[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:28 dist-test-slave-1x4g krb5kdc[1102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:28 dist-test-slave-1x4g krb5kdc[1102](info): set up 2 sockets
Aug 07 05:04:28 dist-test-slave-1x4g krb5kdc[1102](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 05:04:29.159823 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.008s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:29 dist-test-slave-1x4g krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:29.286252 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39837
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:44027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:29.396785 1115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:29.397079 1115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:29.397151 1115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:29.401208 1115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:29.401278 1115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:29.401312 1115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:29.401363 1115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:29.401407 1115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:29.406364 1115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:39837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:29.407408 1115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:29.408447 1115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:29.417410 1123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:29.417423 1121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:29.417428 1120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:29.417479 1115 server_base.cc:1034] running on GCE node
I20260807 05:04:29.418154 1115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:29.419288 1115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:29.420488 1115 hybrid_clock.cc:648] HybridClock initialized: now 1786079069420439 us; error 61 us; skew 500 ppm
Aug 07 05:04:29 dist-test-slave-1x4g krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079069, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:29.423468 1115 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:29.424302 1115 webserver.cc:460] Webserver started at http://127.1.17.254:43945/ using document root <none> and password file <none>
I20260807 05:04:29.424933 1115 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:29.425028 1115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:29.425266 1115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:29.427258 1115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "83002d130863473cae20e0f6e663d743"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "bcbe49cd120a0846a47cde9b17da66fe"
server_key_iv: "e8aae7aec20128d9f05ce747879e4a54"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.427809 1115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "83002d130863473cae20e0f6e663d743"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "bcbe49cd120a0846a47cde9b17da66fe"
server_key_iv: "e8aae7aec20128d9f05ce747879e4a54"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.431510 1115 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:04:29.433981 1130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:29.435194 1115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:29.435333 1115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "83002d130863473cae20e0f6e663d743"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "bcbe49cd120a0846a47cde9b17da66fe"
server_key_iv: "e8aae7aec20128d9f05ce747879e4a54"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.435446 1115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:29.449767 1115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:29.452507 1115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:29.452716 1115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:29.459899 1115 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:39837
I20260807 05:04:29.459949 1183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:39837 every 8 connection(s)
I20260807 05:04:29.460842 1115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:29.462528 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 1115
I20260807 05:04:29.462679 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:29.463740 1184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:29.464005 1095 external_mini_cluster.cc:1428] Setting key 969463e73820226c8e56f4b13df04cd4
I20260807 05:04:29.470391 1184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743: Bootstrap starting.
I20260807 05:04:29.472544 1184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:29.473414 1184 log.cc:824] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:29.475235 1184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743: No bootstrap required, opened a new log
I20260807 05:04:29.478829 1184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83002d130863473cae20e0f6e663d743" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39837 } }
I20260807 05:04:29.479070 1184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:29.479167 1184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83002d130863473cae20e0f6e663d743, State: Initialized, Role: FOLLOWER
I20260807 05:04:29.479611 1184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83002d130863473cae20e0f6e663d743" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39837 } }
I20260807 05:04:29.479768 1184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:29.479857 1184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:29.479981 1184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 05:04:29 dist-test-slave-1x4g krb5kdc[1102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:29.481195 1184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83002d130863473cae20e0f6e663d743" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39837 } }
I20260807 05:04:29.481594 1184 leader_election.cc:302] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83002d130863473cae20e0f6e663d743; no voters:
I20260807 05:04:29.481977 1184 leader_election.cc:288] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:29.482089 1189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:29.482334 1189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [term 1 LEADER]: Becoming Leader. State: Replica: 83002d130863473cae20e0f6e663d743, State: Running, Role: LEADER
I20260807 05:04:29.482679 1189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83002d130863473cae20e0f6e663d743" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39837 } }
I20260807 05:04:29.483317 1184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:29.484499 1191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83002d130863473cae20e0f6e663d743. Latest consensus state: current_term: 1 leader_uuid: "83002d130863473cae20e0f6e663d743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83002d130863473cae20e0f6e663d743" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39837 } } }
I20260807 05:04:29.484642 1191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:29.484985 1190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83002d130863473cae20e0f6e663d743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83002d130863473cae20e0f6e663d743" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39837 } } }
I20260807 05:04:29.485121 1190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:29.486048 1198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:29.488592 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:29.467030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.1.17.254:39837)
0807 05:04:29.467422 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:29.467441 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:29.468580 (+ 1139us) server_negotiation.cc:407] Connection header received
0807 05:04:29.471737 (+ 3157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:29.471751 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:29.472026 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:29.472321 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:29.474490 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.475492 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.476700 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.477222 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.477407 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:29.482038 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:29.482046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:29.482061 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:29.482096 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:29.484420 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:29.485144 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:29.485153 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:29.485158 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:29.485309 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:29.485771 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:29.485774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:29.485776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:29.486409 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:29.486640 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:29.487352 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 05:04:29.487632 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":124,"threads_started":1}
I20260807 05:04:29.489301 1198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:29.494630 1198 catalog_manager.cc:1347] Generated new cluster ID: aa2ee94c608a4a1989107fc1d0f549c2
I20260807 05:04:29.494704 1198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:29.504742 1198 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:29.505676 1198 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:29.517537 1198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83002d130863473cae20e0f6e663d743: Generated new TSK 0
I20260807 05:04:29.518299 1198 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:29.549371 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39837
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:44027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:29.659299 1212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:29.659580 1212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:29.659654 1212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:29.663692 1212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:29.663779 1212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:29.663868 1212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:29.669565 1212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:29.670568 1212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:29.671561 1212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:29.681017 1220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:29.681034 1217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:29.681058 1218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:29.681521 1212 server_base.cc:1034] running on GCE node
I20260807 05:04:29.681893 1212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:29.682473 1212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:29.683646 1212 hybrid_clock.cc:648] HybridClock initialized: now 1786079069683634 us; error 40 us; skew 500 ppm
Aug 07 05:04:29 dist-test-slave-1x4g krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079069, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:29.686601 1212 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:29.687506 1212 webserver.cc:460] Webserver started at http://127.1.17.193:37199/ using document root <none> and password file <none>
I20260807 05:04:29.688063 1212 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:29.688122 1212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:29.688393 1212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:29.690474 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "458a5a2c72f74bdab8bc18e3ee6257ce"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "2dbf8919572a9a42ad0f8ba8e7a88bf0"
server_key_iv: "653a71cf8f8b891ccc467d3d822e6b08"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.691041 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "458a5a2c72f74bdab8bc18e3ee6257ce"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "2dbf8919572a9a42ad0f8ba8e7a88bf0"
server_key_iv: "653a71cf8f8b891ccc467d3d822e6b08"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.694780 1212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 05:04:29.697227 1227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:29.698407 1212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:29.698544 1212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "458a5a2c72f74bdab8bc18e3ee6257ce"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "2dbf8919572a9a42ad0f8ba8e7a88bf0"
server_key_iv: "653a71cf8f8b891ccc467d3d822e6b08"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.698653 1212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:29.729228 1212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:29.732376 1212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:29.732570 1212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:29.733249 1212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:29.734086 1212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:29.734170 1212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:29.734236 1212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:29.734284 1212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:29.743432 1212 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:39473
I20260807 05:04:29.743467 1344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:39473 every 8 connection(s)
I20260807 05:04:29.744342 1212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:29.745401 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 1212
I20260807 05:04:29.745492 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:29.745839 1095 external_mini_cluster.cc:1428] Setting key 0795a3337d00b0688725a182cd82a1da
Aug 07 05:04:29 dist-test-slave-1x4g krb5kdc[1102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079069, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:04:29.759234 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:29.746299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:32811 (local address 127.1.17.254:39837)
0807 05:04:29.746445 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:29.746450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:29.747497 (+ 1047us) server_negotiation.cc:407] Connection header received
0807 05:04:29.749004 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:29.749007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:29.749059 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:29.749138 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:29.750309 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.751013 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.752214 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.752677 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.752813 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:29.755664 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:29.755667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:29.755669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:29.755692 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:29.757242 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:29.757704 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:29.757707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:29.757708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:29.757780 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:29.758130 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:29.758133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:29.758135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:29.758323 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:29.758433 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:29.758913 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 05:04:29.759050 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:04:29.760177 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:29.746789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39837 (local address 127.1.17.193:32811)
0807 05:04:29.747364 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:29.747408 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:29.748739 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:29.749316 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:29.749323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:29.749678 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:29.750165 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:29.750178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.751179 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.751182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:29.752067 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:29.752076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.752890 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.752893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:29.753068 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:29.753707 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:29.753722 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:29.755497 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:29.757351 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:29.757356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:29.757367 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:29.757619 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:29.757857 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:29.757859 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:29.757860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:29.758032 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:29.758433 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:29.758440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:29.758683 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 05:04:29.758870 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 05:04:29.759193 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":141,"threads_started":1}
I20260807 05:04:29.761552 1345 heartbeater.cc:347] Connected to a master server at 127.1.17.254:39837
I20260807 05:04:29.761821 1345 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:29.762398 1345 heartbeater.cc:511] Master 127.1.17.254:39837 requested a full tablet report, sending...
I20260807 05:04:29.764037 1148 ts_manager.cc:194] Registered new tserver with Master: 458a5a2c72f74bdab8bc18e3ee6257ce (127.1.17.193:39473)
I20260807 05:04:29.765295 1148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:32811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:29.777051 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39837
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:44027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:29.890144 1350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:29.890405 1350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:29.890446 1350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:29.894537 1350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:29.894585 1350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:29.894658 1350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:29.899755 1350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:29.900740 1350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:29.901810 1350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:29.911698 1359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:29.911710 1356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:29.911991 1350 server_base.cc:1034] running on GCE node
W20260807 05:04:29.912034 1357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:29.912411 1350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:29.913087 1350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:29.914263 1350 hybrid_clock.cc:648] HybridClock initialized: now 1786079069914234 us; error 41 us; skew 500 ppm
Aug 07 05:04:29 dist-test-slave-1x4g krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079069, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:29.917023 1350 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:29.917805 1350 webserver.cc:460] Webserver started at http://127.1.17.194:44971/ using document root <none> and password file <none>
I20260807 05:04:29.918414 1350 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:29.918485 1350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:29.918644 1350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:29.920629 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "ee1253857c2b4a8faaa20e3254aef9f3"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "5e2c078ffa772421badc34576ca0f55f"
server_key_iv: "e41af7c891dd24a782d989d378fec5be"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.921204 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "ee1253857c2b4a8faaa20e3254aef9f3"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "5e2c078ffa772421badc34576ca0f55f"
server_key_iv: "e41af7c891dd24a782d989d378fec5be"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.924707 1350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:04:29.927111 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:29.928294 1350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:29.928413 1350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "ee1253857c2b4a8faaa20e3254aef9f3"
format_stamp: "Formatted at 2026-08-07 05:04:29 on dist-test-slave-1x4g"
server_key: "5e2c078ffa772421badc34576ca0f55f"
server_key_iv: "e41af7c891dd24a782d989d378fec5be"
server_key_version: "encryptionkey@0"
I20260807 05:04:29.928524 1350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:29.941179 1350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:29.944259 1350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:29.944434 1350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:29.945084 1350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:29.945847 1350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:29.945892 1350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:29.945957 1350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:29.945989 1350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:29.954622 1350 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:46357
I20260807 05:04:29.954661 1484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:46357 every 8 connection(s)
I20260807 05:04:29.955472 1350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:29.963827 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 1350
I20260807 05:04:29.963963 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:29.964262 1095 external_mini_cluster.cc:1428] Setting key 74062da5d05d0e0b90f61e7d468adf75
Aug 07 05:04:29 dist-test-slave-1x4g krb5kdc[1102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079069, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:29.969826 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:29.957288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:43071 (local address 127.1.17.254:39837)
0807 05:04:29.957434 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:29.957439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:29.958310 (+ 871us) server_negotiation.cc:407] Connection header received
0807 05:04:29.959607 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:29.959610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:29.959656 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:29.959728 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:29.960917 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.961575 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.962680 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.962880 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.962980 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:29.965883 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:29.965888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:29.965891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:29.965942 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:29.967594 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:29.968145 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:29.968150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:29.968153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:29.968247 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:29.968628 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:29.968630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:29.968631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:29.968854 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:29.968950 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:29.969607 (+ 657us) server_negotiation.cc:299] Negotiation successful
0807 05:04:29.969707 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:04:29.970723 1487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:29.957703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39837 (local address 127.1.17.194:43071)
0807 05:04:29.958163 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:29.958205 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:29.959416 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:29.959954 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:29.959962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:29.960300 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:29.960758 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:29.960797 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.961690 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.961693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:29.962556 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:29.962565 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:29.963040 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:29.963042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:29.963186 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:29.963710 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:29.963735 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:29.965509 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:29.967710 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:29.967715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:29.967726 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:29.968026 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:29.968338 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:29.968344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:29.968346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:29.968534 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:29.968963 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:29.968973 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:29.969292 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 05:04:29.969567 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 05:04:29.969894 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":126,"threads_started":1}
I20260807 05:04:29.971915 1485 heartbeater.cc:347] Connected to a master server at 127.1.17.254:39837
I20260807 05:04:29.972168 1485 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:29.972751 1485 heartbeater.cc:511] Master 127.1.17.254:39837 requested a full tablet report, sending...
I20260807 05:04:29.974026 1148 ts_manager.cc:194] Registered new tserver with Master: ee1253857c2b4a8faaa20e3254aef9f3 (127.1.17.194:46357)
I20260807 05:04:29.974681 1148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:43071
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:29.994028 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39837
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:44027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:30.105990 1490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:30.106251 1490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:30.106292 1490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:30.110548 1490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:30.110600 1490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:30.110718 1490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:30.115931 1490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:30.117007 1490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:30.118022 1490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:30.127821 1496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:30.127821 1495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:30.128036 1498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:30.128026 1490 server_base.cc:1034] running on GCE node
I20260807 05:04:30.128561 1490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:30.129237 1490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:30.130401 1490 hybrid_clock.cc:648] HybridClock initialized: now 1786079070130387 us; error 26 us; skew 500 ppm
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079070, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:30.133260 1490 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:30.134147 1490 webserver.cc:460] Webserver started at http://127.1.17.195:37077/ using document root <none> and password file <none>
I20260807 05:04:30.134706 1490 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:30.134796 1490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:30.135033 1490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:30.137220 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "1a9592494af84bd9a5f942a908f339f2"
format_stamp: "Formatted at 2026-08-07 05:04:30 on dist-test-slave-1x4g"
server_key: "59e1d8d65a36c6bf62fddc6cd3ccf509"
server_key_iv: "c2163b3bad60e1fb8efee113529e83fb"
server_key_version: "encryptionkey@0"
I20260807 05:04:30.137816 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "1a9592494af84bd9a5f942a908f339f2"
format_stamp: "Formatted at 2026-08-07 05:04:30 on dist-test-slave-1x4g"
server_key: "59e1d8d65a36c6bf62fddc6cd3ccf509"
server_key_iv: "c2163b3bad60e1fb8efee113529e83fb"
server_key_version: "encryptionkey@0"
I20260807 05:04:30.141806 1490 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:04:30.144153 1505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:30.145344 1490 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:30.145474 1490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "1a9592494af84bd9a5f942a908f339f2"
format_stamp: "Formatted at 2026-08-07 05:04:30 on dist-test-slave-1x4g"
server_key: "59e1d8d65a36c6bf62fddc6cd3ccf509"
server_key_iv: "c2163b3bad60e1fb8efee113529e83fb"
server_key_version: "encryptionkey@0"
I20260807 05:04:30.145576 1490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:30.154838 1490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:30.157567 1490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:30.157753 1490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:30.158296 1490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:30.159078 1490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:30.159149 1490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:30.159216 1490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:30.159269 1490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:30.168327 1490 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:35047
I20260807 05:04:30.168354 1622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:35047 every 8 connection(s)
I20260807 05:04:30.169236 1490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:30.170389 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 1490
I20260807 05:04:30.170529 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:30.170799 1095 external_mini_cluster.cc:1428] Setting key 73cbf2fc701cec9548d7f646f9e6df23
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079070, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:30.182874 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:30.171034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:49501 (local address 127.1.17.254:39837)
0807 05:04:30.171190 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:30.171195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:30.172194 (+ 999us) server_negotiation.cc:407] Connection header received
0807 05:04:30.173958 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:30.173961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:30.174015 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:30.174090 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:30.175210 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.175752 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.177001 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.177232 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.177336 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:30.179608 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:30.179611 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:30.179614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:30.179645 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:30.180926 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:30.181406 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:30.181409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:30.181411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:30.181496 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:30.181839 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:30.181844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:30.181845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:30.182029 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:30.182133 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:30.182597 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 05:04:30.182713 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:04:30.183638 1625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:30.171512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39837 (local address 127.1.17.195:49501)
0807 05:04:30.172059 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:30.172102 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:30.173759 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:30.174260 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:30.174267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:30.174602 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:30.175071 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:30.175084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.175886 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.175890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:30.176867 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:30.176877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.177387 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.177391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:30.177555 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:30.178045 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:30.178061 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:30.179455 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:30.181043 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:30.181048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:30.181062 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:30.181303 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:30.181584 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:30.181588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:30.181590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:30.181724 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:30.182145 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:30.182153 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:30.182373 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 05:04:30.182554 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 05:04:30.182888 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":119,"threads_started":1}
I20260807 05:04:30.184746 1623 heartbeater.cc:347] Connected to a master server at 127.1.17.254:39837
I20260807 05:04:30.185091 1623 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:30.185665 1623 heartbeater.cc:511] Master 127.1.17.254:39837 requested a full tablet report, sending...
I20260807 05:04:30.186654 1147 ts_manager.cc:194] Registered new tserver with Master: 1a9592494af84bd9a5f942a908f339f2 (127.1.17.195:35047)
I20260807 05:04:30.187222 1147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:49501
I20260807 05:04:30.197016 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:30.225227 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:30.214709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60188 (local address 127.1.17.193:39473)
0807 05:04:30.215109 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:30.215127 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:30.215152 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:04:30.215203 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:30.215208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:30.215396 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:30.215519 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:30.216209 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.217076 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.217984 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.218356 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.218436 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:30.221195 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:30.221205 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:30.221219 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:30.221243 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:30.223323 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:30.223772 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:30.223778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:30.223780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:30.223872 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:30.224155 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:30.224158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:30.224160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:30.224424 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:30.224666 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:30.224859 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 05:04:30.225034 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":176,"threads_started":1}
W20260807 05:04:30.226307 1255 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60188
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:30.254936 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:30.246424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.1.17.193:39473)
0807 05:04:30.246582 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:30.246587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:30.246708 (+ 121us) server_negotiation.cc:407] Connection header received
0807 05:04:30.246895 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:30.246898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:30.246940 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:30.247035 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:30.247757 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.248282 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.249257 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:30.249490 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:30.249595 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:30.251980 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:30.251983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:30.251985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:30.252007 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:30.253452 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:30.253810 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:30.253813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:30.253814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:30.253912 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:30.254158 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:30.254160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:30.254161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:30.254317 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:30.254414 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:30.254611 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 05:04:30.254705 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:04:30.256000 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 1212
I20260807 05:04:30.264381 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 1350
I20260807 05:04:30.271116 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 1490
I20260807 05:04:30.276991 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 1115
2026-08-07T05:04:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2196 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1637](info): set up 2 sockets
Aug 07 05:04:30 dist-test-slave-1x4g krb5kdc[1637](info): commencing operation
krb5kdc: starting...
W20260807 05:04:31.333329 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:31 dist-test-slave-1x4g krb5kdc[1637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:31.458103 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43725
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:31.568547 1653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:31.568823 1653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:31.568864 1653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:31.572614 1653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:31.572666 1653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:31.572685 1653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:31.572700 1653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:31.572715 1653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:31.577786 1653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:31.578832 1653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:31.579810 1653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:31.588063 1661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:31.588119 1659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:31.588202 1653 server_base.cc:1034] running on GCE node
W20260807 05:04:31.588119 1658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:31.588982 1653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:31.590054 1653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:31.591226 1653 hybrid_clock.cc:648] HybridClock initialized: now 1786079071591196 us; error 42 us; skew 500 ppm
Aug 07 05:04:31 dist-test-slave-1x4g krb5kdc[1637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:31.600481 1653 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:31.601863 1653 webserver.cc:460] Webserver started at http://127.1.17.254:42083/ using document root <none> and password file <none>
I20260807 05:04:31.602696 1653 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:31.602803 1653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:31.603113 1653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:31.606076 1653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "432acc65cee349dd82bdd5f160839be4"
format_stamp: "Formatted at 2026-08-07 05:04:31 on dist-test-slave-1x4g"
server_key: "978bcb84602f9dcb71d569a881b3fba8"
server_key_iv: "2da6b40a58bfa2ab9780a73edaab91c8"
server_key_version: "encryptionkey@0"
I20260807 05:04:31.606791 1653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "432acc65cee349dd82bdd5f160839be4"
format_stamp: "Formatted at 2026-08-07 05:04:31 on dist-test-slave-1x4g"
server_key: "978bcb84602f9dcb71d569a881b3fba8"
server_key_iv: "2da6b40a58bfa2ab9780a73edaab91c8"
server_key_version: "encryptionkey@0"
I20260807 05:04:31.611815 1653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 05:04:31.614938 1668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:31.616655 1653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:31.616830 1653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "432acc65cee349dd82bdd5f160839be4"
format_stamp: "Formatted at 2026-08-07 05:04:31 on dist-test-slave-1x4g"
server_key: "978bcb84602f9dcb71d569a881b3fba8"
server_key_iv: "2da6b40a58bfa2ab9780a73edaab91c8"
server_key_version: "encryptionkey@0"
I20260807 05:04:31.617003 1653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:31.634207 1653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:31.637665 1653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:31.637904 1653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:31.645901 1653 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:43725
I20260807 05:04:31.646207 1720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:43725 every 8 connection(s)
I20260807 05:04:31.646926 1653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:31.649804 1721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:31.654101 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 1653
I20260807 05:04:31.654196 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:31.654484 1095 external_mini_cluster.cc:1428] Setting key bda1e1ae4a05b7e15bff4382ab99d182
I20260807 05:04:31.655580 1721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4: Bootstrap starting.
I20260807 05:04:31.658252 1721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:31.659062 1721 log.cc:824] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:31.660959 1721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4: No bootstrap required, opened a new log
Aug 07 05:04:31 dist-test-slave-1x4g krb5kdc[1637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:31.664403 1721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432acc65cee349dd82bdd5f160839be4" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43725 } }
I20260807 05:04:31.664652 1721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:31.664714 1721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432acc65cee349dd82bdd5f160839be4, State: Initialized, Role: FOLLOWER
I20260807 05:04:31.665264 1721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432acc65cee349dd82bdd5f160839be4" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43725 } }
I20260807 05:04:31.665483 1721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:31.665561 1721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:31.665661 1721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:31.666500 1721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432acc65cee349dd82bdd5f160839be4" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43725 } }
I20260807 05:04:31.666838 1721 leader_election.cc:302] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 432acc65cee349dd82bdd5f160839be4; no voters:
I20260807 05:04:31.667165 1721 leader_election.cc:288] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:31.667268 1726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:31.667480 1726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [term 1 LEADER]: Becoming Leader. State: Replica: 432acc65cee349dd82bdd5f160839be4, State: Running, Role: LEADER
I20260807 05:04:31.667765 1726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432acc65cee349dd82bdd5f160839be4" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43725 } }
I20260807 05:04:31.668442 1721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:31.669407 1728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 432acc65cee349dd82bdd5f160839be4. Latest consensus state: current_term: 1 leader_uuid: "432acc65cee349dd82bdd5f160839be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432acc65cee349dd82bdd5f160839be4" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43725 } } }
I20260807 05:04:31.669579 1728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:31.669989 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:31.655875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.1.17.254:43725)
0807 05:04:31.656276 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:31.656294 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:31.656336 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:04:31.657273 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:31.657297 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:31.657695 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:31.658016 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:31.659277 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:31.660240 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:31.661199 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:31.661672 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:31.661846 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:31.664303 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:31.664308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:31.664321 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:31.664347 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:31.666239 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:31.666715 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:31.666720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:31.666724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:31.666833 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:31.667134 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:31.667138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:31.667140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:31.667669 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:31.668441 (+ 772us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:31.668622 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 05:04:31.668970 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260807 05:04:31.669966 1727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "432acc65cee349dd82bdd5f160839be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432acc65cee349dd82bdd5f160839be4" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43725 } } }
I20260807 05:04:31.670083 1727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4 [sys.catalog]: This master's current role is: LEADER
W20260807 05:04:31.673538 1741 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:04:31.673627 1741 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:04:31.673714 1735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:31.674456 1735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:31.680022 1735 catalog_manager.cc:1347] Generated new cluster ID: 18e07486e7ec45c0b907028026ad50d2
I20260807 05:04:31.680088 1735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:31.689146 1735 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:31.689990 1735 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:31.701719 1735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 432acc65cee349dd82bdd5f160839be4: Generated new TSK 0
I20260807 05:04:31.702430 1735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:31.739485 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43725
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:31.852661 1747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:31.852953 1747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:31.853035 1747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:31.857028 1747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:31.857102 1747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:31.857191 1747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:31.862185 1747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:31.863262 1747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:31.864289 1747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:31.873777 1755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:31.873840 1752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:31.873782 1753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:31.874122 1747 server_base.cc:1034] running on GCE node
I20260807 05:04:31.874583 1747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:31.875211 1747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:31.876422 1747 hybrid_clock.cc:648] HybridClock initialized: now 1786079071876396 us; error 34 us; skew 500 ppm
Aug 07 05:04:31 dist-test-slave-1x4g krb5kdc[1637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:31.879261 1747 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:31.880117 1747 webserver.cc:460] Webserver started at http://127.1.17.193:39413/ using document root <none> and password file <none>
I20260807 05:04:31.880672 1747 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:31.880800 1747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:31.881079 1747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:31.883083 1747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "3ed28c0a21994ba9a2699dbdba62b565"
format_stamp: "Formatted at 2026-08-07 05:04:31 on dist-test-slave-1x4g"
server_key: "d4be91b0fa1bf7117e186fe29912628e"
server_key_iv: "1c2b8658c90e86cd01220a51e8f9bb77"
server_key_version: "encryptionkey@0"
I20260807 05:04:31.883684 1747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "3ed28c0a21994ba9a2699dbdba62b565"
format_stamp: "Formatted at 2026-08-07 05:04:31 on dist-test-slave-1x4g"
server_key: "d4be91b0fa1bf7117e186fe29912628e"
server_key_iv: "1c2b8658c90e86cd01220a51e8f9bb77"
server_key_version: "encryptionkey@0"
I20260807 05:04:31.887504 1747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:04:31.889981 1763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:31.891235 1747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:31.891373 1747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "3ed28c0a21994ba9a2699dbdba62b565"
format_stamp: "Formatted at 2026-08-07 05:04:31 on dist-test-slave-1x4g"
server_key: "d4be91b0fa1bf7117e186fe29912628e"
server_key_iv: "1c2b8658c90e86cd01220a51e8f9bb77"
server_key_version: "encryptionkey@0"
I20260807 05:04:31.891503 1747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:31.904618 1747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:31.907509 1747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:31.907697 1747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:31.908257 1747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:31.909152 1747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:31.909226 1747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:31.909294 1747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:31.909345 1747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:31.918772 1747 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:40415
I20260807 05:04:31.918800 1880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:40415 every 8 connection(s)
I20260807 05:04:31.919626 1747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:31.926110 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 1747
I20260807 05:04:31.926204 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:31.926488 1095 external_mini_cluster.cc:1428] Setting key fe94bb9ad031dd3b543245c8b33848a4
Aug 07 05:04:31 dist-test-slave-1x4g krb5kdc[1637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:31.933907 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:31.921484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:36269 (local address 127.1.17.254:43725)
0807 05:04:31.921669 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:31.921673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:31.922543 (+ 870us) server_negotiation.cc:407] Connection header received
0807 05:04:31.924034 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:31.924037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:31.924092 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:31.924177 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:31.925454 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:31.926000 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:31.927414 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:31.927734 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:31.927828 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:31.930376 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:31.930378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:31.930380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:31.930402 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:31.931831 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:31.932327 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:31.932332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:31.932334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:31.932412 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:31.932749 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:31.932754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:31.932755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:31.932990 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:31.933117 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:31.933603 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 05:04:31.933731 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 05:04:31.934700 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:31.921938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43725 (local address 127.1.17.193:36269)
0807 05:04:31.922396 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:31.922438 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:31.923804 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:31.924355 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:31.924362 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:31.924702 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:31.925291 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:31.925305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:31.926133 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:31.926137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:31.927291 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:31.927301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:31.927868 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:31.927872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:31.928061 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:31.928678 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:31.928694 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:31.930200 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:31.931952 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:31.931958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:31.931972 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:31.932221 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:31.932509 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:31.932512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:31.932514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:31.932653 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:31.933118 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:31.933126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:31.933369 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 05:04:31.933561 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 05:04:31.933873 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260807 05:04:31.936096 1881 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43725
I20260807 05:04:31.936327 1881 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:04:31.936918 1881 heartbeater.cc:511] Master 127.1.17.254:43725 requested a full tablet report, sending...
I20260807 05:04:31.938457 1681 ts_manager.cc:194] Registered new tserver with Master: 3ed28c0a21994ba9a2699dbdba62b565 (127.1.17.193:40415)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:04:31.939990 1681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:36269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:31.956370 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43725
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:32.067368 1886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:32.067602 1886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:32.067643 1886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:32.071727 1886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:32.071779 1886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:32.071834 1886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:32.076824 1886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:32.077728 1886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:32.078668 1886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:32.087878 1891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:32.087952 1892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:32.087970 1886 server_base.cc:1034] running on GCE node
W20260807 05:04:32.087878 1894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:32.088557 1886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:32.089175 1886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:32.090358 1886 hybrid_clock.cc:648] HybridClock initialized: now 1786079072090344 us; error 26 us; skew 500 ppm
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:32.093148 1886 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:32.093894 1886 webserver.cc:460] Webserver started at http://127.1.17.194:39411/ using document root <none> and password file <none>
I20260807 05:04:32.094475 1886 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:32.094547 1886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:32.094712 1886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:32.096722 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "ad1bd8868daa4088ba10f9d3586b4e40"
format_stamp: "Formatted at 2026-08-07 05:04:32 on dist-test-slave-1x4g"
server_key: "0506ecaf45687ec2bff286083be30c3a"
server_key_iv: "53adb60b597a2cc99de0ffee7ccdaa28"
server_key_version: "encryptionkey@0"
I20260807 05:04:32.097288 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "ad1bd8868daa4088ba10f9d3586b4e40"
format_stamp: "Formatted at 2026-08-07 05:04:32 on dist-test-slave-1x4g"
server_key: "0506ecaf45687ec2bff286083be30c3a"
server_key_iv: "53adb60b597a2cc99de0ffee7ccdaa28"
server_key_version: "encryptionkey@0"
I20260807 05:04:32.100682 1886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:04:32.103005 1901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:32.104202 1886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:32.104295 1886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "ad1bd8868daa4088ba10f9d3586b4e40"
format_stamp: "Formatted at 2026-08-07 05:04:32 on dist-test-slave-1x4g"
server_key: "0506ecaf45687ec2bff286083be30c3a"
server_key_iv: "53adb60b597a2cc99de0ffee7ccdaa28"
server_key_version: "encryptionkey@0"
I20260807 05:04:32.104384 1886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:32.118398 1886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:32.121665 1886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:32.121831 1886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:32.122469 1886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:32.123342 1886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:32.123387 1886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:32.123425 1886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:32.123448 1886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:32.132081 1886 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:37881
I20260807 05:04:32.132107 2016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:37881 every 8 connection(s)
I20260807 05:04:32.133018 1886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:32.142632 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 1886
I20260807 05:04:32.142752 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:32.143059 1095 external_mini_cluster.cc:1428] Setting key 2f2cc6856f4254e895d8ac2211c92610
I20260807 05:04:32.146924 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.134747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57145 (local address 127.1.17.254:43725)
0807 05:04:32.134884 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.134889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.135763 (+ 874us) server_negotiation.cc:407] Connection header received
0807 05:04:32.137144 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.137147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.137191 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.137263 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:32.138512 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.139049 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.140205 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.140485 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.140570 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.143161 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:32.143164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:32.143167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:32.143193 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:32.144736 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.145288 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.145295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.145298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.145382 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.145773 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.145777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.145778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.145994 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:32.146113 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.146602 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.146732 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:04:32.147722 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.135157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43725 (local address 127.1.17.194:57145)
0807 05:04:32.135615 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:32.135655 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:32.136960 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:32.137429 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:32.137435 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:32.137873 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:32.138327 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.138339 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.139177 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.139180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.140070 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.140079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.140616 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.140619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.140760 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.141396 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:32.141412 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:32.142976 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:32.144898 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:32.144911 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:32.144926 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:32.145181 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:32.145484 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:32.145488 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:32.145490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:32.145662 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:32.146114 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:32.146121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:32.146361 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 05:04:32.146554 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 05:04:32.146881 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":129,"threads_started":1}
I20260807 05:04:32.148943 2017 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43725
I20260807 05:04:32.149214 2017 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:32.149817 2017 heartbeater.cc:511] Master 127.1.17.254:43725 requested a full tablet report, sending...
I20260807 05:04:32.151048 1681 ts_manager.cc:194] Registered new tserver with Master: ad1bd8868daa4088ba10f9d3586b4e40 (127.1.17.194:37881)
I20260807 05:04:32.151846 1681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:57145
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:32.172147 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43725
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:43857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:32.282752 2022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:32.282958 2022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:32.283013 2022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:32.286942 2022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:32.286990 2022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:32.287081 2022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:32.291994 2022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:43857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:32.292948 2022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:32.293897 2022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:32.303498 2029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:32.303539 2027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:32.303550 2031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:32.303723 2022 server_base.cc:1034] running on GCE node
I20260807 05:04:32.304207 2022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:32.304865 2022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:32.306041 2022 hybrid_clock.cc:648] HybridClock initialized: now 1786079072306015 us; error 44 us; skew 500 ppm
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:32.308948 2022 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:32.309774 2022 webserver.cc:460] Webserver started at http://127.1.17.195:40675/ using document root <none> and password file <none>
I20260807 05:04:32.310329 2022 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:32.310420 2022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:32.310647 2022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:32.312592 2022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "1b976d79dbc146b7b547ec2523de2d07"
format_stamp: "Formatted at 2026-08-07 05:04:32 on dist-test-slave-1x4g"
server_key: "098aa09758232f03c309c09cb1af1158"
server_key_iv: "a44248f383e4df5813ce8ea931f9af89"
server_key_version: "encryptionkey@0"
I20260807 05:04:32.313203 2022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "1b976d79dbc146b7b547ec2523de2d07"
format_stamp: "Formatted at 2026-08-07 05:04:32 on dist-test-slave-1x4g"
server_key: "098aa09758232f03c309c09cb1af1158"
server_key_iv: "a44248f383e4df5813ce8ea931f9af89"
server_key_version: "encryptionkey@0"
I20260807 05:04:32.316706 2022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:04:32.319083 2038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:32.320252 2022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:32.320510 2022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "1b976d79dbc146b7b547ec2523de2d07"
format_stamp: "Formatted at 2026-08-07 05:04:32 on dist-test-slave-1x4g"
server_key: "098aa09758232f03c309c09cb1af1158"
server_key_iv: "a44248f383e4df5813ce8ea931f9af89"
server_key_version: "encryptionkey@0"
I20260807 05:04:32.320657 2022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:32.332690 2022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:32.336175 2022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:32.336393 2022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:32.337041 2022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:32.337929 2022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:32.338006 2022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:32.338083 2022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:32.338126 2022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:32.348202 2022 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:36521
I20260807 05:04:32.348217 2154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:36521 every 8 connection(s)
I20260807 05:04:32.349159 2022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:32.358695 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 2022
I20260807 05:04:32.358829 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:32.359158 1095 external_mini_cluster.cc:1428] Setting key 23a08abd72090529e923eab69b853b72
I20260807 05:04:32.362617 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.350956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:51455 (local address 127.1.17.254:43725)
0807 05:04:32.351125 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.351128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.351849 (+ 721us) server_negotiation.cc:407] Connection header received
0807 05:04:32.353263 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.353267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.353325 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.353431 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:32.354552 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.355104 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.356292 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.356528 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.356668 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.359122 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:32.359125 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:32.359127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:32.359160 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:32.360611 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.361233 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.361237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.361238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.361322 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.361657 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.361660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.361661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.361824 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:32.361920 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.362360 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.362464 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 05:04:32.363333 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.351264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43725 (local address 127.1.17.195:51455)
0807 05:04:32.351698 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:32.351736 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:32.353060 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:32.353602 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:32.353609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:32.353936 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:32.354396 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.354408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.355246 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.355251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.356154 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.356163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.356706 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.356710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.356933 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.357435 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:32.357453 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:32.358963 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:32.360718 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:32.360723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:32.360737 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:32.361123 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:32.361412 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:32.361416 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:32.361417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:32.361559 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:32.361921 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:32.361928 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:32.362144 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 05:04:32.362306 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 05:04:32.362592 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
I20260807 05:04:32.364439 2155 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43725
I20260807 05:04:32.364689 2155 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:32.365223 2155 heartbeater.cc:511] Master 127.1.17.254:43725 requested a full tablet report, sending...
I20260807 05:04:32.366286 1681 ts_manager.cc:194] Registered new tserver with Master: 1b976d79dbc146b7b547ec2523de2d07 (127.1.17.195:36521)
I20260807 05:04:32.366930 1681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:51455
I20260807 05:04:32.373147 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:04:32.384068 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.376835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53170 (local address 127.1.17.254:43725)
0807 05:04:32.376995 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.377001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.377242 (+ 241us) server_negotiation.cc:407] Connection header received
0807 05:04:32.377318 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.377321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.377364 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.377453 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:32.378128 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.378653 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.379433 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.379658 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.379750 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.381105 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:32.381108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:32.381110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:32.381135 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:32.382447 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.382821 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.382823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.382824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.382930 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.383205 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.383208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.383209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.383432 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:32.383554 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.383773 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.383943 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:04:32.388713 1681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:04:32.392052 1681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:32.406611 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.400442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:40415 (local address 127.0.0.1:44552)
0807 05:04:32.401987 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:32.402002 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:32.402087 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:32.402515 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:32.402521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:32.402546 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:32.402673 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.402679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.403896 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.403902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.405095 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.405104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.406118 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.406121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.406224 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.406260 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 05:04:32.406365 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 05:04:32.406470 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260807 05:04:32.406685 2168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.400560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44552 (local address 127.1.17.193:40415)
0807 05:04:32.401154 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.401170 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.402090 (+ 920us) server_negotiation.cc:407] Connection header received
0807 05:04:32.402177 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.402181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.402344 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.402506 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:32.402835 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.403768 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.405207 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.405791 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.405903 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.406054 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.406434 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.406545 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":110,"threads_started":1}
I20260807 05:04:32.407310 2166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.401504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:37881 (local address 127.0.0.1:36040)
0807 05:04:32.402230 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:32.402239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:32.402287 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:32.402950 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:32.402954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:32.402973 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:32.403099 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.403105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.404484 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.404487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.405763 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.405774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.406919 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.406922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.407038 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.407061 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:04:32.407131 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:04:32.407194 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":114,"thread_start_us":90,"threads_started":1}
I20260807 05:04:32.407758 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.401650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36040 (local address 127.1.17.194:37881)
0807 05:04:32.402525 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.402537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.402559 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:04:32.402629 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.402634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.402816 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.402961 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:32.403227 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.404349 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.405892 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.406630 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.406729 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.406858 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.407530 (+ 672us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.407622 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":171,"threads_started":1}
I20260807 05:04:32.407789 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.400585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:36521 (local address 127.0.0.1:41970)
0807 05:04:32.401736 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:32.401815 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:32.401958 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:32.402802 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:32.402805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:32.402820 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:32.402940 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.402946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.404484 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.404487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.405763 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.405774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.407249 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.407252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.407501 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.407529 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 05:04:32.407589 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 05:04:32.407659 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":74,"thread_start_us":86,"threads_started":1}
I20260807 05:04:32.408363 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.400746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41970 (local address 127.1.17.195:36521)
0807 05:04:32.401056 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.401070 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.401906 (+ 836us) server_negotiation.cc:407] Connection header received
0807 05:04:32.402141 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.402147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.402344 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.402506 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:32.403087 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.404328 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.405910 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.406461 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.406554 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.408041 (+ 1487us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.408097 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.408187 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":123,"threads_started":1}
I20260807 05:04:32.410013 1815 tablet_service.cc:1467] Processing CreateTablet for tablet 64b2d59c60da4714b6b96b492c3ef2af (DEFAULT_TABLE table=test-table [id=edba3d59724347dcbaae0678c51c7140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:32.410667 1951 tablet_service.cc:1467] Processing CreateTablet for tablet 64b2d59c60da4714b6b96b492c3ef2af (DEFAULT_TABLE table=test-table [id=edba3d59724347dcbaae0678c51c7140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:32.410660 2087 tablet_service.cc:1467] Processing CreateTablet for tablet 64b2d59c60da4714b6b96b492c3ef2af (DEFAULT_TABLE table=test-table [id=edba3d59724347dcbaae0678c51c7140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:32.411180 1815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b2d59c60da4714b6b96b492c3ef2af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:32.411631 1951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b2d59c60da4714b6b96b492c3ef2af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:32.411640 2087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64b2d59c60da4714b6b96b492c3ef2af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:32.417636 2172 tablet_bootstrap.cc:492] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07: Bootstrap starting.
I20260807 05:04:32.418934 2174 tablet_bootstrap.cc:492] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40: Bootstrap starting.
I20260807 05:04:32.419787 2172 tablet_bootstrap.cc:654] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:32.421167 2172 log.cc:824] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:32.421233 2174 tablet_bootstrap.cc:654] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:32.422115 2174 log.cc:824] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:32.422195 2173 tablet_bootstrap.cc:492] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565: Bootstrap starting.
I20260807 05:04:32.423645 2172 tablet_bootstrap.cc:492] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07: No bootstrap required, opened a new log
I20260807 05:04:32.423839 2172 ts_tablet_manager.cc:1397] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:04:32.424100 2174 tablet_bootstrap.cc:492] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40: No bootstrap required, opened a new log
I20260807 05:04:32.424248 2173 tablet_bootstrap.cc:654] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:32.424273 2174 ts_tablet_manager.cc:1397] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 05:04:32.425477 2173 log.cc:824] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:32.427626 2173 tablet_bootstrap.cc:492] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565: No bootstrap required, opened a new log
I20260807 05:04:32.427809 2173 ts_tablet_manager.cc:1397] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 05:04:32.427822 2172 raft_consensus.cc:348] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.428128 2172 raft_consensus.cc:374] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:32.428185 2172 raft_consensus.cc:729] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b976d79dbc146b7b547ec2523de2d07, State: Initialized, Role: FOLLOWER
I20260807 05:04:32.428040 2174 raft_consensus.cc:348] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.428285 2174 raft_consensus.cc:374] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:32.428337 2174 raft_consensus.cc:729] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1bd8868daa4088ba10f9d3586b4e40, State: Initialized, Role: FOLLOWER
I20260807 05:04:32.428642 2172 consensus_queue.cc:260] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.428726 2174 consensus_queue.cc:260] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.429368 2155 heartbeater.cc:503] Master 127.1.17.254:43725 was elected leader, sending a full tablet report...
I20260807 05:04:32.429697 2174 ts_tablet_manager.cc:1428] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:04:32.429697 2172 ts_tablet_manager.cc:1428] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:04:32.430060 2017 heartbeater.cc:503] Master 127.1.17.254:43725 was elected leader, sending a full tablet report...
I20260807 05:04:32.432384 2173 raft_consensus.cc:348] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.432619 2173 raft_consensus.cc:374] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:32.432674 2173 raft_consensus.cc:729] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed28c0a21994ba9a2699dbdba62b565, State: Initialized, Role: FOLLOWER
I20260807 05:04:32.433647 2173 consensus_queue.cc:260] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.434597 1881 heartbeater.cc:503] Master 127.1.17.254:43725 was elected leader, sending a full tablet report...
I20260807 05:04:32.434778 2173 ts_tablet_manager.cc:1428] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260807 05:04:32.488044 2180 raft_consensus.cc:482] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:32.488227 2180 raft_consensus.cc:504] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.489401 2180 leader_election.cc:288] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b976d79dbc146b7b547ec2523de2d07 (127.1.17.195:36521), ad1bd8868daa4088ba10f9d3586b4e40 (127.1.17.194:37881)
I20260807 05:04:32.493288 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.489809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:51145 (local address 127.1.17.194:37881)
0807 05:04:32.489915 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.489920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.490189 (+ 269us) server_negotiation.cc:407] Connection header received
0807 05:04:32.490305 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.490308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.490356 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.490439 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:32.490720 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.491143 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.492248 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.492672 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.492762 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.492860 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.493058 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.493158 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:04:32.493292 2183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.489653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:37881 (local address 127.1.17.193:51145)
0807 05:04:32.490091 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:32.490106 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:32.490199 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:32.490461 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:32.490463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:32.490477 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:32.490620 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.490626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.491275 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.491277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.492123 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.492132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.492819 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.492823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.492921 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.492953 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 05:04:32.493019 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 05:04:32.493101 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":6,"thread_start_us":123,"threads_started":1}
I20260807 05:04:32.493337 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.489723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:49505 (local address 127.1.17.195:36521)
0807 05:04:32.489866 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.489872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.490036 (+ 164us) server_negotiation.cc:407] Connection header received
0807 05:04:32.490132 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.490138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.490195 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.490271 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:32.490572 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.491103 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.492285 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.492702 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.492857 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.492909 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.493145 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.493225 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:04:32.493311 2181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.489563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:36521 (local address 127.1.17.193:49505)
0807 05:04:32.489914 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:32.489928 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:32.490043 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:32.490298 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:32.490301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:32.490317 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:32.490454 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.490461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.491209 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.491214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.492126 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:32.492135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.492866 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.492872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:32.492976 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.492993 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:04:32.493091 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 05:04:32.493171 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":66,"thread_start_us":105,"threads_started":1}
I20260807 05:04:32.493965 2108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b2d59c60da4714b6b96b492c3ef2af" candidate_uuid: "3ed28c0a21994ba9a2699dbdba62b565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b976d79dbc146b7b547ec2523de2d07" is_pre_election: true
I20260807 05:04:32.494031 1971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b2d59c60da4714b6b96b492c3ef2af" candidate_uuid: "3ed28c0a21994ba9a2699dbdba62b565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" is_pre_election: true
I20260807 05:04:32.494297 2108 raft_consensus.cc:2454] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed28c0a21994ba9a2699dbdba62b565 in term 0.
I20260807 05:04:32.494372 1971 raft_consensus.cc:2454] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed28c0a21994ba9a2699dbdba62b565 in term 0.
I20260807 05:04:32.494869 1766 leader_election.cc:302] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b976d79dbc146b7b547ec2523de2d07, 3ed28c0a21994ba9a2699dbdba62b565, ad1bd8868daa4088ba10f9d3586b4e40; no voters:
I20260807 05:04:32.495127 2180 raft_consensus.cc:2781] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:32.495231 2180 raft_consensus.cc:482] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:32.495285 2180 raft_consensus.cc:3037] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:32.496497 2180 raft_consensus.cc:504] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.496857 2180 leader_election.cc:288] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [CANDIDATE]: Term 1 election: Requested vote from peers 1b976d79dbc146b7b547ec2523de2d07 (127.1.17.195:36521), ad1bd8868daa4088ba10f9d3586b4e40 (127.1.17.194:37881)
I20260807 05:04:32.497174 2108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b2d59c60da4714b6b96b492c3ef2af" candidate_uuid: "3ed28c0a21994ba9a2699dbdba62b565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b976d79dbc146b7b547ec2523de2d07"
I20260807 05:04:32.497293 2108 raft_consensus.cc:3037] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:32.498270 2108 raft_consensus.cc:2454] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed28c0a21994ba9a2699dbdba62b565 in term 1.
I20260807 05:04:32.498528 1971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64b2d59c60da4714b6b96b492c3ef2af" candidate_uuid: "3ed28c0a21994ba9a2699dbdba62b565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1bd8868daa4088ba10f9d3586b4e40"
I20260807 05:04:32.498672 1971 raft_consensus.cc:3037] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:32.498672 1764 leader_election.cc:302] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b976d79dbc146b7b547ec2523de2d07, 3ed28c0a21994ba9a2699dbdba62b565; no voters:
I20260807 05:04:32.498878 2180 raft_consensus.cc:2781] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:32.499123 2180 raft_consensus.cc:686] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed28c0a21994ba9a2699dbdba62b565, State: Running, Role: LEADER
I20260807 05:04:32.499521 2180 consensus_queue.cc:237] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } }
I20260807 05:04:32.500006 1971 raft_consensus.cc:2454] T 64b2d59c60da4714b6b96b492c3ef2af P ad1bd8868daa4088ba10f9d3586b4e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed28c0a21994ba9a2699dbdba62b565 in term 1.
I20260807 05:04:32.503026 1681 catalog_manager.cc:5495] T 64b2d59c60da4714b6b96b492c3ef2af P 3ed28c0a21994ba9a2699dbdba62b565 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ed28c0a21994ba9a2699dbdba62b565 (127.1.17.193). New cstate: current_term: 1 leader_uuid: "3ed28c0a21994ba9a2699dbdba62b565" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ed28c0a21994ba9a2699dbdba62b565" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b976d79dbc146b7b547ec2523de2d07" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad1bd8868daa4088ba10f9d3586b4e40" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37881 } health_report { overall_health: UNKNOWN } } }
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:32.534781 2168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.525761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.1.17.193:40415)
0807 05:04:32.525914 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.525918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.526128 (+ 210us) server_negotiation.cc:407] Connection header received
0807 05:04:32.526224 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.526227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.526271 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.526357 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:32.527064 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.527562 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.528393 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.528609 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.528691 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.531317 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:32.531331 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:32.531348 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:32.531371 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:32.533080 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.533483 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.533489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.533490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.533589 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.533858 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.533862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.533863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.534163 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:32.534337 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.534555 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.534657 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:32.563072 2168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.554525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.1.17.193:40415)
0807 05:04:32.554690 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.554694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.554830 (+ 136us) server_negotiation.cc:407] Connection header received
0807 05:04:32.555018 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.555020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.555079 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.555163 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:32.555850 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.556353 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.557297 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.557521 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.557624 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.559996 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:32.559999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:32.560001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:32.560024 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:32.561348 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.561781 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.561785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.561786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.561862 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.562150 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.562154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.562156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.562328 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:32.562502 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.562785 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.562908 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[1637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:32.589536 2168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:32.580992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44586 (local address 127.1.17.193:40415)
0807 05:04:32.581135 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:32.581143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:32.581198 (+ 55us) server_negotiation.cc:407] Connection header received
0807 05:04:32.581369 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:32.581372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:32.581416 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:32.581518 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:32.582205 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.582737 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.583615 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:32.583839 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:32.583950 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:32.586445 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:32.586448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:32.586450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:32.586479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:32.587922 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.588292 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.588294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.588295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.588368 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:32.588589 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:32.588591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:32.588593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:32.588765 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:32.588929 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:32.589209 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 05:04:32.589307 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 05:04:32.590999 1794 tablet_service.cc:2908] Rejecting scan request for tablet 64b2d59c60da4714b6b96b492c3ef2af: Uninitialized: clean time has not yet been initialized
I20260807 05:04:32.593179 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 1747
W20260807 05:04:32.600214 2156 tablet.cc:2369] T 64b2d59c60da4714b6b96b492c3ef2af P 1b976d79dbc146b7b547ec2523de2d07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:04:32.601505 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 1886
I20260807 05:04:32.608049 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 2022
I20260807 05:04:32.615346 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 1653
2026-08-07T05:04:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2338 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[2198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[2198](info): set up 2 sockets
Aug 07 05:04:32 dist-test-slave-1x4g krb5kdc[2198](info): commencing operation
krb5kdc: starting...
W20260807 05:04:33.662731 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:33 dist-test-slave-1x4g krb5kdc[2198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:33.787987 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36663
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:36663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:33.898931 2211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:33.899231 2211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:33.899318 2211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:33.903251 2211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:33.903321 2211 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 05:04:33.903352 2211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:33.903409 2211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:33.903457 2211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:33.903493 2211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:33.908555 2211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:36663
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:33.909595 2211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:33.910579 2211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:33.919135 2220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:33.919142 2217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:33.919193 2218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:33.919150 2211 server_base.cc:1034] running on GCE node
I20260807 05:04:33.919871 2211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:33.920945 2211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:33.922104 2211 hybrid_clock.cc:648] HybridClock initialized: now 1786079073922086 us; error 30 us; skew 500 ppm
Aug 07 05:04:33 dist-test-slave-1x4g krb5kdc[2198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079073, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:33.928330 2211 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:33.929291 2211 webserver.cc:460] Webserver started at http://127.1.17.254:37087/ using document root <none> and password file <none>
I20260807 05:04:33.929854 2211 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:33.929948 2211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:33.930195 2211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:33.932166 2211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "248fda7759d5441ea017b21f522e7144"
format_stamp: "Formatted at 2026-08-07 05:04:33 on dist-test-slave-1x4g"
server_key: "88922d6455d7a618736a5537de7079b3"
server_key_iv: "de94a0daabb8d6864ca688efc3ca6758"
server_key_version: "encryptionkey@0"
I20260807 05:04:33.932730 2211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "248fda7759d5441ea017b21f522e7144"
format_stamp: "Formatted at 2026-08-07 05:04:33 on dist-test-slave-1x4g"
server_key: "88922d6455d7a618736a5537de7079b3"
server_key_iv: "de94a0daabb8d6864ca688efc3ca6758"
server_key_version: "encryptionkey@0"
I20260807 05:04:33.936267 2211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:33.938623 2228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:33.939734 2211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:33.939880 2211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "248fda7759d5441ea017b21f522e7144"
format_stamp: "Formatted at 2026-08-07 05:04:33 on dist-test-slave-1x4g"
server_key: "88922d6455d7a618736a5537de7079b3"
server_key_iv: "de94a0daabb8d6864ca688efc3ca6758"
server_key_version: "encryptionkey@0"
I20260807 05:04:33.939985 2211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:33.967195 2211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:33.970253 2211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:33.970479 2211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:33.978703 2211 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:36663
I20260807 05:04:33.978725 2294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:36663 every 8 connection(s)
I20260807 05:04:33.979647 2211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:33.982482 2295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:33.984128 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 2211
I20260807 05:04:33.984256 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:33.984524 1095 external_mini_cluster.cc:1428] Setting key a2b8074e7ffd8c3259407f1df45a5399
I20260807 05:04:33.988547 2295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144: Bootstrap starting.
I20260807 05:04:33.990777 2295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:33.991581 2295 log.cc:824] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:04:33 dist-test-slave-1x4g krb5kdc[2198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:33.993944 2295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144: No bootstrap required, opened a new log
I20260807 05:04:33.997391 2295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248fda7759d5441ea017b21f522e7144" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36663 } }
I20260807 05:04:33.997704 2295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:33.997790 2295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 248fda7759d5441ea017b21f522e7144, State: Initialized, Role: FOLLOWER
I20260807 05:04:33.998297 2295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248fda7759d5441ea017b21f522e7144" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36663 } }
I20260807 05:04:33.998469 2295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:33.998602 2295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:33.998720 2295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:33.999626 2295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248fda7759d5441ea017b21f522e7144" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36663 } }
I20260807 05:04:33.999814 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:33.986038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.1.17.254:36663)
0807 05:04:33.986447 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:33.986461 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:33.986496 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:04:33.987134 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:33.987150 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:33.987467 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:33.987800 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:33.988575 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:33.989685 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:33.990540 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:33.991079 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:33.991251 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:33.994115 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:33.994121 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:33.994134 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:33.994160 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:33.996530 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:33.997134 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:33.997140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:33.997144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:33.997283 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:33.997623 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:33.997626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:33.997627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:33.998183 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:33.998361 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:33.998732 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 05:04:33.999034 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":124,"threads_started":1}
I20260807 05:04:34.000008 2295 leader_election.cc:302] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 248fda7759d5441ea017b21f522e7144; no voters:
I20260807 05:04:34.000341 2295 leader_election.cc:288] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:34.000480 2300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:34.000756 2300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [term 1 LEADER]: Becoming Leader. State: Replica: 248fda7759d5441ea017b21f522e7144, State: Running, Role: LEADER
I20260807 05:04:34.001165 2300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248fda7759d5441ea017b21f522e7144" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36663 } }
I20260807 05:04:34.001611 2295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:34.002343 2301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 248fda7759d5441ea017b21f522e7144. Latest consensus state: current_term: 1 leader_uuid: "248fda7759d5441ea017b21f522e7144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248fda7759d5441ea017b21f522e7144" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36663 } } }
I20260807 05:04:34.002575 2301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:34.003011 2309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:34.003260 2302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "248fda7759d5441ea017b21f522e7144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248fda7759d5441ea017b21f522e7144" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36663 } } }
I20260807 05:04:34.003459 2302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:34.006309 2309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:34.012168 2309 catalog_manager.cc:1347] Generated new cluster ID: 18e62d8eca384ab9bfe0374154c0ec3e
I20260807 05:04:34.012250 2309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:34.032244 2309 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:34.033501 2309 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:34.041019 2309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144: Generated new TSK 0
I20260807 05:04:34.041796 2309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:34.075480 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36663
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:34.192637 2323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:34.192909 2323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:34.192953 2323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:34.196573 2323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:34.196637 2323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:34.196696 2323 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:04:34.196729 2323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:34.201517 2323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:34.202495 2323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:34.203871 2323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:34.213231 2328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:34.213195 2331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:34.213202 2329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:34.213546 2323 server_base.cc:1034] running on GCE node
I20260807 05:04:34.213992 2323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:34.214612 2323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:34.215808 2323 hybrid_clock.cc:648] HybridClock initialized: now 1786079074215793 us; error 35 us; skew 500 ppm
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:34.218626 2323 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:34.219542 2323 webserver.cc:460] Webserver started at http://127.1.17.193:40267/ using document root <none> and password file <none>
I20260807 05:04:34.220101 2323 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:34.220198 2323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:34.220448 2323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:34.222527 2323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "418e19f83d9e4254b7da2d5cedf1f661"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "6085bc9c3ee16d0628ee1081ad624283"
server_key_iv: "0c25249ce3bd095bbe80147afef18ecd"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.223121 2323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "418e19f83d9e4254b7da2d5cedf1f661"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "6085bc9c3ee16d0628ee1081ad624283"
server_key_iv: "0c25249ce3bd095bbe80147afef18ecd"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.226917 2323 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:04:34.229355 2338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:34.230476 2323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:34.230609 2323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "418e19f83d9e4254b7da2d5cedf1f661"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "6085bc9c3ee16d0628ee1081ad624283"
server_key_iv: "0c25249ce3bd095bbe80147afef18ecd"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.230736 2323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:34.252436 2323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:34.255108 2323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:34.255300 2323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:34.256590 2323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:34.256693 2323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:34.256760 2323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:34.256822 2323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:34.268275 2323 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:37675
I20260807 05:04:34.268342 2457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:37675 every 8 connection(s)
I20260807 05:04:34.269328 2323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:34.271996 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 2323
I20260807 05:04:34.272121 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:34.272403 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.257452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:41673 (local address 127.1.17.254:36663)
0807 05:04:34.257605 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.257611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.259327 (+ 1716us) server_negotiation.cc:407] Connection header received
0807 05:04:34.260819 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.260825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.260887 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.260977 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:34.262712 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.263373 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.265274 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.265514 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.265597 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.268600 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:34.268604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:34.268607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:34.268645 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:34.270498 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.270945 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.270948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.270949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.271024 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.271328 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.271330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.271331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.271504 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:34.271578 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.272146 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.272282 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:04:34.272413 1095 external_mini_cluster.cc:1428] Setting key 4aaf96b614cb472c02c43aab874868a9
I20260807 05:04:34.273339 2347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.257906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36663 (local address 127.1.17.193:41673)
0807 05:04:34.259122 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.259199 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.260508 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.261199 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.261207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.261760 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:34.262522 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.262545 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.263519 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.263526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.265153 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.265164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.265681 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.265688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.265903 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.266548 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:34.266574 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:34.268415 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:34.270603 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.270607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.270617 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.270857 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.271103 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.271105 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.271106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.271251 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.271591 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:34.271596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:34.271848 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.272077 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.272386 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":169,"threads_started":1}
I20260807 05:04:34.276140 2459 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36663
I20260807 05:04:34.276278 2459 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:34.276837 2459 heartbeater.cc:511] Master 127.1.17.254:36663 requested a full tablet report, sending...
I20260807 05:04:34.278474 2245 ts_manager.cc:194] Registered new tserver with Master: 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675)
I20260807 05:04:34.279839 2245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:41673
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:04:34.285274 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.277180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:51049 (local address 127.1.17.254:36663)
0807 05:04:34.277279 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.277283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.277655 (+ 372us) server_negotiation.cc:407] Connection header received
0807 05:04:34.277758 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.277760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.277803 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.277895 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:34.278455 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.279172 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.280554 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.280933 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.281057 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.282140 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:34.282144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:34.282146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:34.282169 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:34.283618 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.284058 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.284062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.284064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.284142 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.284433 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.284436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.284438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.284601 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:34.284706 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.285007 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.285120 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:04:34.285310 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.277111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36663 (local address 127.1.17.193:51049)
0807 05:04:34.277534 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.277571 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.277668 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.277936 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.277941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.278166 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:34.278310 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.278320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.279349 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.279355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.280378 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.280391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.281064 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.281067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.281188 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.281676 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:34.281687 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:34.281995 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:34.283734 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.283739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.283741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.283975 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.284236 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.284240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.284241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.284331 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.284701 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:34.284704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:34.284815 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.284952 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.285122 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:34.301249 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36663
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:34.411021 2468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:34.411295 2468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:34.411414 2468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:34.415426 2468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:34.415511 2468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:34.415582 2468 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:04:34.415675 2468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:34.421061 2468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:34.422035 2468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:34.423067 2468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:34.432530 2473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:34.432544 2474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:34.432735 2468 server_base.cc:1034] running on GCE node
W20260807 05:04:34.432565 2476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:34.433270 2468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:34.433869 2468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:34.435056 2468 hybrid_clock.cc:648] HybridClock initialized: now 1786079074435036 us; error 39 us; skew 500 ppm
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:34.437916 2468 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:34.438777 2468 webserver.cc:460] Webserver started at http://127.1.17.194:39287/ using document root <none> and password file <none>
I20260807 05:04:34.439302 2468 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:34.439389 2468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:34.439658 2468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:34.441591 2468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "0ffd02afce634a3d996a7dc2a5631d3b"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "cac05c39ac9a3761510577aab3bd62a5"
server_key_iv: "0cf63a997a28f23f80aa4b66614ed403"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.442157 2468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "0ffd02afce634a3d996a7dc2a5631d3b"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "cac05c39ac9a3761510577aab3bd62a5"
server_key_iv: "0cf63a997a28f23f80aa4b66614ed403"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.445852 2468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:04:34.448153 2483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:34.449406 2468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:34.449556 2468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "0ffd02afce634a3d996a7dc2a5631d3b"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "cac05c39ac9a3761510577aab3bd62a5"
server_key_iv: "0cf63a997a28f23f80aa4b66614ed403"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.449666 2468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:34.463117 2468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:34.466228 2468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:34.466423 2468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:34.467875 2468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:34.467944 2468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:34.467999 2468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:34.468046 2468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:34.479249 2468 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:40073
I20260807 05:04:34.479318 2602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:40073 every 8 connection(s)
I20260807 05:04:34.480212 2468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:34.482092 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.468560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:56633 (local address 127.1.17.254:36663)
0807 05:04:34.468728 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.468734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.470098 (+ 1364us) server_negotiation.cc:407] Connection header received
0807 05:04:34.471490 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.471497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.471546 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.471621 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:34.473365 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.473924 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.475289 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.475584 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.475698 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.478529 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:34.478532 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:34.478534 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:34.478557 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:34.480026 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.480622 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.480625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.480627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.480724 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.481156 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.481161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.481164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.481355 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:34.481434 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.481877 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.481976 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:04:34.482856 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.469420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36663 (local address 127.1.17.194:56633)
0807 05:04:34.469906 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.469982 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.471221 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.471861 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.471872 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.472406 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:34.473177 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.473200 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.474063 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.474066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.475132 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.475143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.475738 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.475742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.475925 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.476464 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:34.476480 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:34.478367 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:34.480146 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.480152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.480169 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.480492 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.480857 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.480861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.480862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.481028 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.481465 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:34.481469 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:34.481672 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.481813 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.482046 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":111,"threads_started":1}
I20260807 05:04:34.484059 2603 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36663
I20260807 05:04:34.484201 2603 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:34.484782 2603 heartbeater.cc:511] Master 127.1.17.254:36663 requested a full tablet report, sending...
I20260807 05:04:34.485956 2245 ts_manager.cc:194] Registered new tserver with Master: 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073)
I20260807 05:04:34.486670 2245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:56633
I20260807 05:04:34.487169 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 2468
I20260807 05:04:34.487298 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:34.487607 1095 external_mini_cluster.cc:1428] Setting key e0ea761386b01d4b7b2f5d809997488f
I20260807 05:04:34.494155 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.486452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57563 (local address 127.1.17.254:36663)
0807 05:04:34.486577 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.486581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.486821 (+ 240us) server_negotiation.cc:407] Connection header received
0807 05:04:34.486861 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.486864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.486940 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.487025 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:34.487833 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.488559 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.489436 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.489689 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.489800 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.490941 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:34.490944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:34.490946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:34.490967 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:34.492293 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.492831 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.492834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.492836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.492920 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.493222 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.493225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.493226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.493441 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:34.493518 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.493852 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.493960 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:04:34.494253 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.486391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36663 (local address 127.1.17.194:57563)
0807 05:04:34.486680 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.486696 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.486796 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.487338 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.487341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.487562 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:34.487696 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.487705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.488716 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.488719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.489313 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.489322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.489805 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.489808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.489895 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.490402 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:34.490416 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:34.490788 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:34.492411 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.492417 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.492422 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.492697 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.493016 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.493022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.493027 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.493131 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.493540 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:34.493546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:34.493669 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.493811 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.493994 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:34.516305 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36663
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:34.628458 2612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:34.628736 2612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:34.628835 2612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:34.632685 2612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:34.632766 2612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:34.632853 2612 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:04:34.632931 2612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:34.638296 2612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:36663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:34.639328 2612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:34.640326 2612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:34.649977 2618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:34.649963 2617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:34.649966 2620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:34.650229 2612 server_base.cc:1034] running on GCE node
I20260807 05:04:34.650643 2612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:34.651266 2612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:34.652478 2612 hybrid_clock.cc:648] HybridClock initialized: now 1786079074652456 us; error 32 us; skew 500 ppm
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:34.655369 2612 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:34.656217 2612 webserver.cc:460] Webserver started at http://127.1.17.195:40873/ using document root <none> and password file <none>
I20260807 05:04:34.656740 2612 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:34.656864 2612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:34.657059 2612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:34.658964 2612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "272bc813bd954351bca51bc1223f9898"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "73f164f647682825837c6bd6380051a6"
server_key_iv: "3ae84b877200e86714d6979ed0c42e4e"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.659478 2612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "272bc813bd954351bca51bc1223f9898"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "73f164f647682825837c6bd6380051a6"
server_key_iv: "3ae84b877200e86714d6979ed0c42e4e"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.663142 2612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 05:04:34.665447 2628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:34.666613 2612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:34.666706 2612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "272bc813bd954351bca51bc1223f9898"
format_stamp: "Formatted at 2026-08-07 05:04:34 on dist-test-slave-1x4g"
server_key: "73f164f647682825837c6bd6380051a6"
server_key_iv: "3ae84b877200e86714d6979ed0c42e4e"
server_key_version: "encryptionkey@0"
I20260807 05:04:34.666790 2612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:34.682633 2612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:34.685813 2612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:34.685967 2612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:34.687325 2612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:34.687399 2612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:34.687464 2612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:34.687515 2612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:34.698691 2612 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:41485
I20260807 05:04:34.698904 2747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:41485 every 8 connection(s)
I20260807 05:04:34.699666 2612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:34.701681 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.688052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:46755 (local address 127.1.17.254:36663)
0807 05:04:34.688193 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.688197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.689772 (+ 1575us) server_negotiation.cc:407] Connection header received
0807 05:04:34.691015 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.691019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.691083 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.691175 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:34.692962 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.693496 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.694782 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.695015 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.695132 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.697926 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:34.697929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:34.697931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:34.697955 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:34.699463 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.700051 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.700055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.700056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.700153 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.700545 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.700549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.700551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.700767 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:34.700912 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.701459 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.701560 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:04:34.702493 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 2612
I20260807 05:04:34.702631 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:34.702654 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.689101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36663 (local address 127.1.17.195:46755)
0807 05:04:34.689627 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.689667 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.690779 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.691366 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.691378 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.692036 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:34.692809 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.692835 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.693641 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.693648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.694646 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.694656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.695183 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.695188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.695363 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.695891 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:34.695906 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:34.697747 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:34.699579 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.699587 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.699601 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.699917 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.700262 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.700265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.700266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.700419 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.700948 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:34.700953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:34.701243 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.701389 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.701620 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":132,"threads_started":1}
I20260807 05:04:34.702926 1095 external_mini_cluster.cc:1428] Setting key 59db4edc6d42020fa95641fc122a7b8c
I20260807 05:04:34.703995 2748 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36663
I20260807 05:04:34.704144 2748 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:34.704702 2748 heartbeater.cc:511] Master 127.1.17.254:36663 requested a full tablet report, sending...
I20260807 05:04:34.705781 2245 ts_manager.cc:194] Registered new tserver with Master: 272bc813bd954351bca51bc1223f9898 (127.1.17.195:41485)
I20260807 05:04:34.706605 2245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:46755
I20260807 05:04:34.712355 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.706143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:43123 (local address 127.1.17.254:36663)
0807 05:04:34.706259 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.706263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.706444 (+ 181us) server_negotiation.cc:407] Connection header received
0807 05:04:34.706544 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.706547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.706580 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.706642 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:34.707128 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.707607 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.708607 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.708866 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.708939 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.709780 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:34.709783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:34.709785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:34.709804 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:34.711140 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.711497 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.711500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.711501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.711568 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.711788 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.711790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.711791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.711932 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:34.712003 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.712184 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.712259 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:04:34.712371 2754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.706068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36663 (local address 127.1.17.195:43123)
0807 05:04:34.706357 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.706372 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.706472 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.706686 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.706692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.706896 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:34.707017 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.707024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.707730 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.707733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.708496 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.708505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.708960 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.708962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.709028 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.709378 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:34.709387 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:34.709667 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:34.711235 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.711237 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.711239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.711410 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.711642 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:34.711644 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:34.711645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:34.711716 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:34.712008 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:34.712010 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:34.712078 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.712153 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.712246 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":109,"threads_started":1}
I20260807 05:04:34.716490 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:04:34 dist-test-slave-1x4g krb5kdc[2198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:34.743147 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.734769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.1.17.254:36663)
0807 05:04:34.734887 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.734891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.734902 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:04:34.734936 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.734938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.734975 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.735062 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:34.735727 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.736230 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.737146 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.737365 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.737481 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.739805 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:34.739808 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:34.739810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:34.739831 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:34.741271 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.741668 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.741670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.741672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.741745 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:34.742053 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:34.742056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:34.742057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:34.742254 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:34.742395 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.742742 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.742840 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:04:34.746224 2245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:04:34.749447 2245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:34.764071 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.758697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41485 (local address 127.0.0.1:55414)
0807 05:04:34.759760 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.759770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.759879 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.760377 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.760381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.760399 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:34.760488 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.760494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.761764 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.761767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.762574 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.762584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.763481 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.763485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.763594 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.763637 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.763768 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.763895 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"mutex_wait_us":1,"thread_start_us":158,"threads_started":1}
I20260807 05:04:34.764075 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.758966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.1.17.195:41485)
0807 05:04:34.759289 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.759302 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.759981 (+ 679us) server_negotiation.cc:407] Connection header received
0807 05:04:34.760068 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.760073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.760248 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.760387 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:34.760612 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.761639 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.762701 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.763346 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.763459 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.763627 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.763803 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.763899 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":141,"threads_started":1}
I20260807 05:04:34.764816 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.758013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37675 (local address 127.0.0.1:60642)
0807 05:04:34.759330 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.759344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.759424 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.760042 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.760045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.760059 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:34.760131 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.760135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.761349 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.761352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.762993 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.763005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.764325 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.764328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.764474 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.764502 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.764587 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.764657 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1260,"mutex_wait_us":208,"thread_start_us":100,"threads_started":1}
I20260807 05:04:34.765115 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.758223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60642 (local address 127.1.17.193:37675)
0807 05:04:34.759421 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.759432 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.759451 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:04:34.759548 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.759553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.759736 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.759895 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:34.760254 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.761220 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.763128 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.763972 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.764079 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.764257 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.764869 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.764977 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1106,"thread_start_us":74,"threads_started":1}
I20260807 05:04:34.765564 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.757921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:40073 (local address 127.0.0.1:39430)
0807 05:04:34.758697 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:34.758748 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:34.758880 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:34.759553 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:34.759559 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:34.759585 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:34.759723 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.759730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.761220 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.761224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.762994 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:34.763005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.764307 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.764311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:34.764407 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.765263 (+ 856us) client_negotiation.cc:769] Sending connection context
0807 05:04:34.765339 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:04:34.765426 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":1051,"thread_start_us":70,"threads_started":1}
I20260807 05:04:34.766040 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:34.758037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.1.17.194:40073)
0807 05:04:34.758644 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:34.758657 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:34.759050 (+ 393us) server_negotiation.cc:407] Connection header received
0807 05:04:34.759142 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:34.759148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:34.759379 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:34.759552 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:34.759862 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.761105 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.763137 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:34.763923 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:34.764024 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:34.764459 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:34.765773 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0807 05:04:34.765897 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":115,"threads_started":1}
I20260807 05:04:34.767266 2680 tablet_service.cc:1467] Processing CreateTablet for tablet eedcd297ef384e759d4fd216e8f91db5 (DEFAULT_TABLE table=test-table [id=78afa99a6bbb44cf9a22fe71f67815fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:34.767500 2374 tablet_service.cc:1467] Processing CreateTablet for tablet eedcd297ef384e759d4fd216e8f91db5 (DEFAULT_TABLE table=test-table [id=78afa99a6bbb44cf9a22fe71f67815fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:34.768190 2680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eedcd297ef384e759d4fd216e8f91db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:34.768292 2374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eedcd297ef384e759d4fd216e8f91db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:34.770787 2534 tablet_service.cc:1467] Processing CreateTablet for tablet eedcd297ef384e759d4fd216e8f91db5 (DEFAULT_TABLE table=test-table [id=78afa99a6bbb44cf9a22fe71f67815fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:34.771593 2534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eedcd297ef384e759d4fd216e8f91db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:34.775660 2771 tablet_bootstrap.cc:492] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Bootstrap starting.
I20260807 05:04:34.778139 2771 tablet_bootstrap.cc:654] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:34.779182 2772 tablet_bootstrap.cc:492] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Bootstrap starting.
I20260807 05:04:34.779744 2771 log.cc:824] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:34.780431 2773 tablet_bootstrap.cc:492] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Bootstrap starting.
I20260807 05:04:34.781299 2772 tablet_bootstrap.cc:654] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:34.782177 2772 log.cc:824] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:34.782279 2771 tablet_bootstrap.cc:492] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: No bootstrap required, opened a new log
I20260807 05:04:34.782470 2771 ts_tablet_manager.cc:1397] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 05:04:34.782814 2773 tablet_bootstrap.cc:654] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:34.783510 2773 log.cc:824] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:34.785427 2772 tablet_bootstrap.cc:492] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: No bootstrap required, opened a new log
I20260807 05:04:34.785573 2773 tablet_bootstrap.cc:492] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: No bootstrap required, opened a new log
I20260807 05:04:34.785638 2772 ts_tablet_manager.cc:1397] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 05:04:34.785763 2773 ts_tablet_manager.cc:1397] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:04:34.786718 2771 raft_consensus.cc:348] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:34.786980 2771 raft_consensus.cc:374] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:34.787060 2771 raft_consensus.cc:729] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 272bc813bd954351bca51bc1223f9898, State: Initialized, Role: FOLLOWER
I20260807 05:04:34.787539 2771 consensus_queue.cc:260] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:34.788771 2748 heartbeater.cc:503] Master 127.1.17.254:36663 was elected leader, sending a full tablet report...
I20260807 05:04:34.789265 2771 ts_tablet_manager.cc:1428] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 05:04:34.789561 2773 raft_consensus.cc:348] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:34.789742 2773 raft_consensus.cc:374] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:34.789774 2773 raft_consensus.cc:729] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418e19f83d9e4254b7da2d5cedf1f661, State: Initialized, Role: FOLLOWER
I20260807 05:04:34.789669 2772 raft_consensus.cc:348] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:34.789903 2772 raft_consensus.cc:374] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:34.789974 2772 raft_consensus.cc:729] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd02afce634a3d996a7dc2a5631d3b, State: Initialized, Role: FOLLOWER
I20260807 05:04:34.790143 2773 consensus_queue.cc:260] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:34.790373 2772 consensus_queue.cc:260] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:34.790740 2459 heartbeater.cc:503] Master 127.1.17.254:36663 was elected leader, sending a full tablet report...
I20260807 05:04:34.790982 2773 ts_tablet_manager.cc:1428] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260807 05:04:34.791993 2603 heartbeater.cc:503] Master 127.1.17.254:36663 was elected leader, sending a full tablet report...
I20260807 05:04:34.793201 2772 ts_tablet_manager.cc:1428] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260807 05:04:34.950690 2749 tablet.cc:2369] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:04:34.981419 2604 tablet.cc:2369] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:04:35.020748 2460 tablet.cc:2369] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:04:35.112531 2779 raft_consensus.cc:482] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:35.112869 2779 raft_consensus.cc:504] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.114140 2779 leader_election.cc:288] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675), 272bc813bd954351bca51bc1223f9898 (127.1.17.195:41485)
I20260807 05:04:35.118386 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.114402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41485 (local address 127.1.17.194:44599)
0807 05:04:35.114906 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.114920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.115025 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.115313 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.115317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.115331 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:35.115416 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.115424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.116292 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.116298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.117241 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.117251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.117933 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.117940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.118052 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.118076 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.118151 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.118231 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":27,"thread_start_us":77,"threads_started":1}
I20260807 05:04:35.118432 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.114584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:44599 (local address 127.1.17.195:41485)
0807 05:04:35.114717 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.114721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.115013 (+ 292us) server_negotiation.cc:407] Connection header received
0807 05:04:35.115113 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.115116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.115167 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.115236 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:35.115521 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.116148 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.117367 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.117798 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.117894 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.117936 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.118201 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.118290 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:04:35.118481 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.114515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:59699 (local address 127.1.17.193:37675)
0807 05:04:35.114649 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.114654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.114864 (+ 210us) server_negotiation.cc:407] Connection header received
0807 05:04:35.115001 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.115004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.115086 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.115168 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:35.115510 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.116208 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.117478 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.117909 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.118032 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.118081 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.118269 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.118345 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:04:35.118861 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.114393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37675 (local address 127.1.17.194:59699)
0807 05:04:35.114757 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.114780 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.114881 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.115193 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.115196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.115212 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:35.115353 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.115360 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.116336 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.116339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.117333 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.117346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.118040 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.118043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.118111 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.118159 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.118236 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.118301 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":18,"thread_start_us":107,"threads_started":1}
I20260807 05:04:35.119184 2700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eedcd297ef384e759d4fd216e8f91db5" candidate_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272bc813bd954351bca51bc1223f9898" is_pre_election: true
I20260807 05:04:35.119499 2700 raft_consensus.cc:2454] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd02afce634a3d996a7dc2a5631d3b in term 0.
I20260807 05:04:35.119525 2404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eedcd297ef384e759d4fd216e8f91db5" candidate_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418e19f83d9e4254b7da2d5cedf1f661" is_pre_election: true
I20260807 05:04:35.119866 2404 raft_consensus.cc:2454] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd02afce634a3d996a7dc2a5631d3b in term 0.
I20260807 05:04:35.120056 2484 leader_election.cc:302] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd02afce634a3d996a7dc2a5631d3b, 272bc813bd954351bca51bc1223f9898; no voters:
I20260807 05:04:35.120332 2779 raft_consensus.cc:2781] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:35.120424 2779 raft_consensus.cc:482] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:35.120471 2779 raft_consensus.cc:3037] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.121621 2779 raft_consensus.cc:504] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.121943 2779 leader_election.cc:288] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [CANDIDATE]: Term 1 election: Requested vote from peers 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675), 272bc813bd954351bca51bc1223f9898 (127.1.17.195:41485)
I20260807 05:04:35.122296 2700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eedcd297ef384e759d4fd216e8f91db5" candidate_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272bc813bd954351bca51bc1223f9898"
I20260807 05:04:35.122318 2404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eedcd297ef384e759d4fd216e8f91db5" candidate_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418e19f83d9e4254b7da2d5cedf1f661"
I20260807 05:04:35.122431 2700 raft_consensus.cc:3037] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.122447 2404 raft_consensus.cc:3037] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.123842 2404 raft_consensus.cc:2454] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd02afce634a3d996a7dc2a5631d3b in term 1.
I20260807 05:04:35.124191 2487 leader_election.cc:302] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd02afce634a3d996a7dc2a5631d3b, 418e19f83d9e4254b7da2d5cedf1f661; no voters:
I20260807 05:04:35.124325 2700 raft_consensus.cc:2454] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd02afce634a3d996a7dc2a5631d3b in term 1.
I20260807 05:04:35.124357 2779 raft_consensus.cc:2781] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:35.124579 2779 raft_consensus.cc:686] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 LEADER]: Becoming Leader. State: Replica: 0ffd02afce634a3d996a7dc2a5631d3b, State: Running, Role: LEADER
I20260807 05:04:35.124991 2779 consensus_queue.cc:237] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.128206 2244 catalog_manager.cc:5495] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194). New cstate: current_term: 1 leader_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } health_report { overall_health: UNKNOWN } } }
I20260807 05:04:35.217561 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.213701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.1.17.194:40073)
0807 05:04:35.213857 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.213862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.213878 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:04:35.214102 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.214105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.214164 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.214274 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:35.214550 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.215081 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.215981 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.216204 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.216312 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.216672 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.217130 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.217258 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.217344 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.217404 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:04:35.223368 2700 raft_consensus.cc:1261] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Refusing update from remote peer 0ffd02afce634a3d996a7dc2a5631d3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:35.223368 2404 raft_consensus.cc:1261] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Refusing update from remote peer 0ffd02afce634a3d996a7dc2a5631d3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:35.224141 2779 consensus_queue.cc:1035] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:35.224347 2782 consensus_queue.cc:1035] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:35.244107 2244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:35.244288 2244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:35.247838 2244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144: Sending DeleteTablet for 3 replicas of tablet eedcd297ef384e759d4fd216e8f91db5
I20260807 05:04:35.248674 2534 tablet_service.cc:1514] Processing DeleteTablet for tablet eedcd297ef384e759d4fd216e8f91db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:35 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:39430
I20260807 05:04:35.248898 2680 tablet_service.cc:1514] Processing DeleteTablet for tablet eedcd297ef384e759d4fd216e8f91db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:35 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:55414
I20260807 05:04:35.249193 2374 tablet_service.cc:1514] Processing DeleteTablet for tablet eedcd297ef384e759d4fd216e8f91db5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:35 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60642
I20260807 05:04:35.249267 2796 tablet_replica.cc:330] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: stopping tablet replica
I20260807 05:04:35.249330 2245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:04:35.249572 2796 raft_consensus.cc:2229] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 05:04:35.249711 2245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:35.249871 2796 raft_consensus.cc:2258] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:35.250030 2797 tablet_replica.cc:330] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: stopping tablet replica
I20260807 05:04:35.250317 2797 raft_consensus.cc:2229] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:35.250623 2795 tablet_replica.cc:330] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: stopping tablet replica
I20260807 05:04:35.250670 2797 raft_consensus.cc:2258] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:35.250888 2795 raft_consensus.cc:2229] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:35.251258 2795 raft_consensus.cc:2258] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:35.252353 2797 ts_tablet_manager.cc:1905] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:35.252566 2795 ts_tablet_manager.cc:1905] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:35.255543 2797 ts_tablet_manager.cc:1918] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:04:35.255714 2797 log.cc:1196] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/eedcd297ef384e759d4fd216e8f91db5
I20260807 05:04:35.256114 2797 ts_tablet_manager.cc:1939] T eedcd297ef384e759d4fd216e8f91db5 P 418e19f83d9e4254b7da2d5cedf1f661: Deleting consensus metadata
I20260807 05:04:35.256318 2795 ts_tablet_manager.cc:1918] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:04:35.256408 2795 log.cc:1196] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/eedcd297ef384e759d4fd216e8f91db5
I20260807 05:04:35.256579 2374 tablet_service.cc:1467] Processing CreateTablet for tablet 0faecc1f3c564dfb96320e96fe66cad8 (DEFAULT_TABLE table=test-table [id=eb4f4795c4ab4f479583924a5ae9df38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:35.253666 2796 ts_tablet_manager.cc:1905] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:35.256742 2534 tablet_service.cc:1467] Processing CreateTablet for tablet 0faecc1f3c564dfb96320e96fe66cad8 (DEFAULT_TABLE table=test-table [id=eb4f4795c4ab4f479583924a5ae9df38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:35.256889 2374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0faecc1f3c564dfb96320e96fe66cad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:35.257019 2534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0faecc1f3c564dfb96320e96fe66cad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:35.259128 2772 tablet_bootstrap.cc:492] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: Bootstrap starting.
I20260807 05:04:35.259210 2680 tablet_service.cc:1467] Processing CreateTablet for tablet 0faecc1f3c564dfb96320e96fe66cad8 (DEFAULT_TABLE table=test-table [id=eb4f4795c4ab4f479583924a5ae9df38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:35.259300 2796 ts_tablet_manager.cc:1918] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:04:35.259392 2796 log.cc:1196] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/eedcd297ef384e759d4fd216e8f91db5
I20260807 05:04:35.259457 2680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0faecc1f3c564dfb96320e96fe66cad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:35.259675 2796 ts_tablet_manager.cc:1939] T eedcd297ef384e759d4fd216e8f91db5 P 272bc813bd954351bca51bc1223f9898: Deleting consensus metadata
I20260807 05:04:35.260097 2772 tablet_bootstrap.cc:654] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:35.256740 2795 ts_tablet_manager.cc:1939] T eedcd297ef384e759d4fd216e8f91db5 P 0ffd02afce634a3d996a7dc2a5631d3b: Deleting consensus metadata
I20260807 05:04:35.261730 2771 tablet_bootstrap.cc:492] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: Bootstrap starting.
I20260807 05:04:35.261826 2229 catalog_manager.cc:4841] TS 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073): tablet eedcd297ef384e759d4fd216e8f91db5 (table test-table [id=78afa99a6bbb44cf9a22fe71f67815fe]) successfully deleted
I20260807 05:04:35.262107 2229 catalog_manager.cc:4841] TS 272bc813bd954351bca51bc1223f9898 (127.1.17.195:41485): tablet eedcd297ef384e759d4fd216e8f91db5 (table test-table [id=78afa99a6bbb44cf9a22fe71f67815fe]) successfully deleted
I20260807 05:04:35.262583 2773 tablet_bootstrap.cc:492] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: Bootstrap starting.
I20260807 05:04:35.262701 2771 tablet_bootstrap.cc:654] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:35.263477 2773 tablet_bootstrap.cc:654] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:35.263445 2232 catalog_manager.cc:4841] TS 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675): tablet eedcd297ef384e759d4fd216e8f91db5 (table test-table [id=78afa99a6bbb44cf9a22fe71f67815fe]) successfully deleted
I20260807 05:04:35.264106 2772 tablet_bootstrap.cc:492] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: No bootstrap required, opened a new log
I20260807 05:04:35.264190 2772 ts_tablet_manager.cc:1397] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 05:04:35.264266 2771 tablet_bootstrap.cc:492] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: No bootstrap required, opened a new log
I20260807 05:04:35.264345 2771 ts_tablet_manager.cc:1397] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:35.264648 2772 raft_consensus.cc:348] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.264770 2772 raft_consensus.cc:374] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:35.264838 2772 raft_consensus.cc:729] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd02afce634a3d996a7dc2a5631d3b, State: Initialized, Role: FOLLOWER
I20260807 05:04:35.264840 2771 raft_consensus.cc:348] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.264951 2771 raft_consensus.cc:374] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:35.264953 2772 consensus_queue.cc:260] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.265010 2771 raft_consensus.cc:729] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 272bc813bd954351bca51bc1223f9898, State: Initialized, Role: FOLLOWER
I20260807 05:04:35.265141 2771 consensus_queue.cc:260] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.265208 2772 ts_tablet_manager.cc:1428] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:35.265426 2771 ts_tablet_manager.cc:1428] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:35.265470 2773 tablet_bootstrap.cc:492] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: No bootstrap required, opened a new log
I20260807 05:04:35.265540 2773 ts_tablet_manager.cc:1397] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 05:04:35.266155 2773 raft_consensus.cc:348] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.266291 2773 raft_consensus.cc:374] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:35.266323 2773 raft_consensus.cc:729] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418e19f83d9e4254b7da2d5cedf1f661, State: Initialized, Role: FOLLOWER
I20260807 05:04:35.266436 2773 consensus_queue.cc:260] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.266729 2773 ts_tablet_manager.cc:1428] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:35.662792 2778 raft_consensus.cc:482] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:35.662964 2778 raft_consensus.cc:504] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.664160 2778 leader_election.cc:288] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073), 272bc813bd954351bca51bc1223f9898 (127.1.17.195:41485)
I20260807 05:04:35.669689 2777 raft_consensus.cc:482] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:35.669763 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.664588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:58773 (local address 127.1.17.194:40073)
0807 05:04:35.664746 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.664751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.664970 (+ 219us) server_negotiation.cc:407] Connection header received
0807 05:04:35.665105 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.665108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.665166 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.665216 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:35.665713 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.666505 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.668346 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.669032 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.669159 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.669234 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.669534 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.669611 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 05:04:35.669847 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.665015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:36607 (local address 127.1.17.195:41485)
0807 05:04:35.665242 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.665246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.665261 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:04:35.665306 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.665308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.665346 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.665420 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:35.665711 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.666505 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.668346 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.669042 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.669159 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.669222 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.669534 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.669611 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":65,"threads_started":1}
I20260807 05:04:35.669777 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.664495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:40073 (local address 127.1.17.193:58773)
0807 05:04:35.664772 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.664834 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.664969 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.665363 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.665365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.665379 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:35.665585 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.665592 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.666622 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.666625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.668218 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.668226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.669172 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.669176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.669335 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.669413 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.669505 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.669582 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":306,"thread_start_us":95,"threads_started":1}
I20260807 05:04:35.669849 2777 raft_consensus.cc:504] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.670538 2554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faecc1f3c564dfb96320e96fe66cad8" candidate_uuid: "418e19f83d9e4254b7da2d5cedf1f661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" is_pre_election: true
I20260807 05:04:35.670792 2554 raft_consensus.cc:2454] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 418e19f83d9e4254b7da2d5cedf1f661 in term 0.
I20260807 05:04:35.670969 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.664867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41485 (local address 127.1.17.193:36607)
0807 05:04:35.665118 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.665131 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.665218 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.665440 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.665442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.665454 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:35.665585 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.665592 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.666622 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.666625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.668214 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.668226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.669172 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.669176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.669331 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.669411 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.669497 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.670792 (+ 1295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":229,"thread_start_us":57,"threads_started":1}
I20260807 05:04:35.671306 2777 leader_election.cc:288] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675), 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073)
I20260807 05:04:35.671257 2339 leader_election.cc:302] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd02afce634a3d996a7dc2a5631d3b, 418e19f83d9e4254b7da2d5cedf1f661; no voters:
I20260807 05:04:35.671558 2778 raft_consensus.cc:2781] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:35.671618 2778 raft_consensus.cc:482] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:35.671664 2778 raft_consensus.cc:3037] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.671923 2700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faecc1f3c564dfb96320e96fe66cad8" candidate_uuid: "418e19f83d9e4254b7da2d5cedf1f661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272bc813bd954351bca51bc1223f9898" is_pre_election: true
I20260807 05:04:35.672052 2700 raft_consensus.cc:2454] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 418e19f83d9e4254b7da2d5cedf1f661 in term 0.
I20260807 05:04:35.672969 2778 raft_consensus.cc:504] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.673336 2778 leader_election.cc:288] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073), 272bc813bd954351bca51bc1223f9898 (127.1.17.195:41485)
I20260807 05:04:35.673714 2700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faecc1f3c564dfb96320e96fe66cad8" candidate_uuid: "418e19f83d9e4254b7da2d5cedf1f661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272bc813bd954351bca51bc1223f9898"
I20260807 05:04:35.673830 2700 raft_consensus.cc:3037] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.673914 2554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faecc1f3c564dfb96320e96fe66cad8" candidate_uuid: "418e19f83d9e4254b7da2d5cedf1f661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd02afce634a3d996a7dc2a5631d3b"
I20260807 05:04:35.674016 2554 raft_consensus.cc:3037] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.675022 2700 raft_consensus.cc:2454] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 418e19f83d9e4254b7da2d5cedf1f661 in term 1.
I20260807 05:04:35.675230 2554 raft_consensus.cc:2454] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 418e19f83d9e4254b7da2d5cedf1f661 in term 1.
I20260807 05:04:35.675477 2339 leader_election.cc:302] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 272bc813bd954351bca51bc1223f9898, 418e19f83d9e4254b7da2d5cedf1f661; no voters:
I20260807 05:04:35.675688 2778 raft_consensus.cc:2781] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:35.675912 2778 raft_consensus.cc:686] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 LEADER]: Becoming Leader. State: Replica: 418e19f83d9e4254b7da2d5cedf1f661, State: Running, Role: LEADER
I20260807 05:04:35.676158 2778 consensus_queue.cc:237] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.676589 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.671480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37675 (local address 127.1.17.195:45887)
0807 05:04:35.671881 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.671895 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.672012 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.672833 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.672839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.672864 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:35.672988 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.672996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.673939 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.673942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.675230 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.675241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.676173 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.676177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.676264 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.676287 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.676333 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.676416 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":95,"threads_started":1}
I20260807 05:04:35.677031 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.671727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:46367 (local address 127.1.17.194:40073)
0807 05:04:35.671840 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.671846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.673036 (+ 1190us) server_negotiation.cc:407] Connection header received
0807 05:04:35.673183 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.673185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.673233 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.673270 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:35.673693 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.674401 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.675826 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.676351 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.676498 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.676541 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.676849 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.676918 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:04:35.677065 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.672280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:45887 (local address 127.1.17.193:37675)
0807 05:04:35.672565 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.672573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.672586 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:04:35.672629 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.672632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.672678 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.672769 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:35.673128 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.673812 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.675370 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.676016 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.676808 (+ 792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.676859 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.676890 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.676945 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":140,"threads_started":1}
I20260807 05:04:35.677271 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.671551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:40073 (local address 127.1.17.195:46367)
0807 05:04:35.672962 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.672976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.673073 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.673429 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.673431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.673449 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:35.673571 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.673578 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.674528 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.674531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.675681 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.675691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.676504 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.676508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.676592 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.676632 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.676726 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.677159 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"thread_start_us":83,"threads_started":1}
I20260807 05:04:35.677659 2404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faecc1f3c564dfb96320e96fe66cad8" candidate_uuid: "272bc813bd954351bca51bc1223f9898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418e19f83d9e4254b7da2d5cedf1f661" is_pre_election: true
I20260807 05:04:35.678397 2554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faecc1f3c564dfb96320e96fe66cad8" candidate_uuid: "272bc813bd954351bca51bc1223f9898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" is_pre_election: true
I20260807 05:04:35.678587 2554 raft_consensus.cc:2379] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 272bc813bd954351bca51bc1223f9898 in current term 1: Already voted for candidate 418e19f83d9e4254b7da2d5cedf1f661 in this term.
I20260807 05:04:35.678870 2246 catalog_manager.cc:5495] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 reported cstate change: term changed from 0 to 1, leader changed from <none> to 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193). New cstate: current_term: 1 leader_uuid: "418e19f83d9e4254b7da2d5cedf1f661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } health_report { overall_health: UNKNOWN } } }
I20260807 05:04:35.679025 2629 leader_election.cc:302] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 272bc813bd954351bca51bc1223f9898; no voters: 0ffd02afce634a3d996a7dc2a5631d3b, 418e19f83d9e4254b7da2d5cedf1f661
I20260807 05:04:35.679257 2777 raft_consensus.cc:2726] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 07 05:04:35 dist-test-slave-1x4g krb5kdc[2198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079073, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:35.735126 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.726321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36663 (local address 127.0.0.1:51126)
0807 05:04:35.726610 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.726627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.726732 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.726975 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.726977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.727356 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:35.727448 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.727455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.728272 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.728275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.728865 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.728874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.729342 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.729345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.729405 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.730116 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:35.730128 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:35.731550 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:35.733404 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:35.733419 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:35.733425 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:35.733656 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:35.733995 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:35.733998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:35.733999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:35.734087 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:35.734506 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:35.734514 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:35.734631 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.734751 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.734902 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":112,"threads_started":1}
I20260807 05:04:35.735193 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.726370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.1.17.254:36663)
0807 05:04:35.726637 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.726644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.726703 (+ 59us) server_negotiation.cc:407] Connection header received
0807 05:04:35.726805 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.726807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.726844 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.726927 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:35.727557 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.728171 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.728991 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.729240 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.729327 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.731717 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:35.731721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:35.731723 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:35.731747 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:35.733286 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:35.733743 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:35.733746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:35.733747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:35.733882 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:35.734165 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:35.734168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:35.734169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:35.734386 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:35.734506 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.734797 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.734971 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":205,"thread_start_us":122,"threads_started":1}
I20260807 05:04:35.739348 2246 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:51126:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 05:04:35.740183 2246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:35.745013 2374 tablet_service.cc:1467] Processing CreateTablet for tablet cab215acf084463bb75022bccf5d1575 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6097e638f7c047b78ab58c0566d00ab1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:04:35.745116 2534 tablet_service.cc:1467] Processing CreateTablet for tablet cab215acf084463bb75022bccf5d1575 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6097e638f7c047b78ab58c0566d00ab1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:04:35.745451 2374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab215acf084463bb75022bccf5d1575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:35.745342 2680 tablet_service.cc:1467] Processing CreateTablet for tablet cab215acf084463bb75022bccf5d1575 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6097e638f7c047b78ab58c0566d00ab1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:04:35.745519 2534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab215acf084463bb75022bccf5d1575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:35.745666 2680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab215acf084463bb75022bccf5d1575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:35.747978 2771 tablet_bootstrap.cc:492] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: Bootstrap starting.
I20260807 05:04:35.748391 2772 tablet_bootstrap.cc:492] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: Bootstrap starting.
I20260807 05:04:35.748898 2771 tablet_bootstrap.cc:654] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:35.749308 2772 tablet_bootstrap.cc:654] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:35.749866 2773 tablet_bootstrap.cc:492] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: Bootstrap starting.
I20260807 05:04:35.750139 2771 tablet_bootstrap.cc:492] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: No bootstrap required, opened a new log
I20260807 05:04:35.750236 2771 ts_tablet_manager.cc:1397] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 05:04:35.750420 2772 tablet_bootstrap.cc:492] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: No bootstrap required, opened a new log
I20260807 05:04:35.750487 2772 ts_tablet_manager.cc:1397] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:35.750694 2773 tablet_bootstrap.cc:654] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:35.750711 2771 raft_consensus.cc:348] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.750814 2771 raft_consensus.cc:374] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:35.750851 2771 raft_consensus.cc:729] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 272bc813bd954351bca51bc1223f9898, State: Initialized, Role: FOLLOWER
I20260807 05:04:35.750844 2772 raft_consensus.cc:348] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.750947 2772 raft_consensus.cc:374] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:35.750978 2772 raft_consensus.cc:729] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd02afce634a3d996a7dc2a5631d3b, State: Initialized, Role: FOLLOWER
I20260807 05:04:35.750985 2771 consensus_queue.cc:260] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.751113 2772 consensus_queue.cc:260] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.751274 2771 ts_tablet_manager.cc:1428] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:04:35.751361 2772 ts_tablet_manager.cc:1428] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:35.751385 2777 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.751534 2777 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:35.751544 2790 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.751665 2790 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:35.752418 2773 tablet_bootstrap.cc:492] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: No bootstrap required, opened a new log
I20260807 05:04:35.752508 2773 ts_tablet_manager.cc:1397] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 05:04:35.753068 2773 raft_consensus.cc:348] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.753175 2773 raft_consensus.cc:374] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:35.753213 2773 raft_consensus.cc:729] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418e19f83d9e4254b7da2d5cedf1f661, State: Initialized, Role: FOLLOWER
I20260807 05:04:35.753338 2773 consensus_queue.cc:260] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.753612 2773 ts_tablet_manager.cc:1428] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:04:35.753592 2778 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.753697 2778 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:35.756263 2777 raft_consensus.cc:482] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:35.756368 2777 raft_consensus.cc:504] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.756763 2777 leader_election.cc:288] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675), 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073)
I20260807 05:04:35.757091 2404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab215acf084463bb75022bccf5d1575" candidate_uuid: "272bc813bd954351bca51bc1223f9898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418e19f83d9e4254b7da2d5cedf1f661" is_pre_election: true
I20260807 05:04:35.757166 2554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab215acf084463bb75022bccf5d1575" candidate_uuid: "272bc813bd954351bca51bc1223f9898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" is_pre_election: true
I20260807 05:04:35.757206 2404 raft_consensus.cc:2454] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 272bc813bd954351bca51bc1223f9898 in term 0.
I20260807 05:04:35.757313 2554 raft_consensus.cc:2454] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 272bc813bd954351bca51bc1223f9898 in term 0.
I20260807 05:04:35.757527 2632 leader_election.cc:302] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 272bc813bd954351bca51bc1223f9898, 418e19f83d9e4254b7da2d5cedf1f661; no voters:
I20260807 05:04:35.757756 2777 raft_consensus.cc:2781] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:35.757833 2777 raft_consensus.cc:482] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:35.757880 2777 raft_consensus.cc:3037] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.759027 2777 raft_consensus.cc:504] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.759363 2777 leader_election.cc:288] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [CANDIDATE]: Term 1 election: Requested vote from peers 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675), 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073)
I20260807 05:04:35.759614 2404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab215acf084463bb75022bccf5d1575" candidate_uuid: "272bc813bd954351bca51bc1223f9898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418e19f83d9e4254b7da2d5cedf1f661"
I20260807 05:04:35.759716 2404 raft_consensus.cc:3037] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.759685 2554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab215acf084463bb75022bccf5d1575" candidate_uuid: "272bc813bd954351bca51bc1223f9898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd02afce634a3d996a7dc2a5631d3b"
I20260807 05:04:35.759807 2554 raft_consensus.cc:3037] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:35.760597 2404 raft_consensus.cc:2454] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 272bc813bd954351bca51bc1223f9898 in term 1.
I20260807 05:04:35.760707 2554 raft_consensus.cc:2454] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 272bc813bd954351bca51bc1223f9898 in term 1.
I20260807 05:04:35.760936 2632 leader_election.cc:302] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 272bc813bd954351bca51bc1223f9898, 418e19f83d9e4254b7da2d5cedf1f661; no voters:
I20260807 05:04:35.761096 2777 raft_consensus.cc:2781] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:35.761267 2777 raft_consensus.cc:686] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [term 1 LEADER]: Becoming Leader. State: Replica: 272bc813bd954351bca51bc1223f9898, State: Running, Role: LEADER
I20260807 05:04:35.761511 2777 consensus_queue.cc:237] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } }
I20260807 05:04:35.762782 2822 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: TxnStatusTablet state changed. Reason: New leader 272bc813bd954351bca51bc1223f9898. Latest consensus state: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.763012 2822 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:04:35.763397 2824 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:04:35.763640 2824 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 05:04:35.763998 2244 catalog_manager.cc:5495] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 reported cstate change: term changed from 0 to 1, leader changed from <none> to 272bc813bd954351bca51bc1223f9898 (127.1.17.195). New cstate: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } health_report { overall_health: HEALTHY } } }
I20260807 05:04:35.779050 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.774805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41485 (local address 127.0.0.1:55420)
0807 05:04:35.775064 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.775078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.775187 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.775569 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.775571 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.775595 (+ 24us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:04:35.775705 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.775709 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.776734 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.776736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.777387 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.777396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.777996 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.777999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.778062 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.778107 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.778807 (+ 700us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.778820 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.778871 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.778946 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260807 05:04:35.779066 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.774929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55420 (local address 127.1.17.195:41485)
0807 05:04:35.775031 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.775035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.775139 (+ 104us) server_negotiation.cc:407] Connection header received
0807 05:04:35.775376 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.775380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.775439 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.775551 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:35.775806 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.776593 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.777521 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.777893 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.777996 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.778234 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.778713 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.778820 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.778884 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.778937 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 05:04:35.782131 2554 raft_consensus.cc:1261] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Refusing update from remote peer 272bc813bd954351bca51bc1223f9898: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:35.782560 2404 raft_consensus.cc:1261] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Refusing update from remote peer 272bc813bd954351bca51bc1223f9898: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:35.782603 2822 consensus_queue.cc:1035] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:35.782991 2822 consensus_queue.cc:1035] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:35.784291 2778 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: TxnStatusTablet state changed. Reason: New leader 272bc813bd954351bca51bc1223f9898. Latest consensus state: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.784399 2778 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:35.784345 2790 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: TxnStatusTablet state changed. Reason: New leader 272bc813bd954351bca51bc1223f9898. Latest consensus state: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.784462 2790 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:35.787221 2778 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.787314 2778 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 418e19f83d9e4254b7da2d5cedf1f661: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:35.788341 2790 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.788455 2790 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 0ffd02afce634a3d996a7dc2a5631d3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:35.788667 2827 mvcc.cc:205] Tried to move back new op lower bound from 7315779894398341120 to 7315779894321799168. Current Snapshot: MvccSnapshot[applied={T|T < 7315779894398341120 or (T in {7315779894398341120})}]
I20260807 05:04:35.789381 2777 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.789485 2777 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:04:35.789535 2822 tablet_replica.cc:439] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "272bc813bd954351bca51bc1223f9898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418e19f83d9e4254b7da2d5cedf1f661" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37675 } } peers { permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 } } peers { permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 } } }
I20260807 05:04:35.789618 2822 tablet_replica.cc:442] T cab215acf084463bb75022bccf5d1575 P 272bc813bd954351bca51bc1223f9898: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:04:35.796625 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.792808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.1.17.193:37675)
0807 05:04:35.792953 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.792958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.793135 (+ 177us) server_negotiation.cc:407] Connection header received
0807 05:04:35.793254 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.793256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.793301 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.793381 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:35.793669 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.794162 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.795316 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.795508 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.795579 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.795849 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.796251 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.796331 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.796440 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.796485 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 05:04:35.797231 2829 mvcc.cc:205] Tried to move back new op lower bound from 7315779894398341120 to 7315779894321799168. Current Snapshot: MvccSnapshot[applied={T|T < 7315779894398341120}]
I20260807 05:04:35.806931 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.803665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:56713 (local address 127.1.17.195:41485)
0807 05:04:35.803817 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.803821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.804015 (+ 194us) server_negotiation.cc:407] Connection header received
0807 05:04:35.804160 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.804164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.804215 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.804294 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:35.804589 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.805132 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.805945 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.806174 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.806274 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.806550 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.806609 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.806685 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.806768 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.806808 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:04:35.807171 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.803583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41485 (local address 127.1.17.193:56713)
0807 05:04:35.803934 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.803946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.804059 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.804307 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.804310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.804317 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:04:35.804472 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.804478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.805252 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.805255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.805826 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.805833 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.806278 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.806282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.806374 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.806412 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.806693 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.806701 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.806758 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.806828 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":92,"threads_started":1}
I20260807 05:04:35.814102 2700 raft_consensus.cc:1261] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Refusing update from remote peer 418e19f83d9e4254b7da2d5cedf1f661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:35.814538 2554 raft_consensus.cc:1261] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Refusing update from remote peer 418e19f83d9e4254b7da2d5cedf1f661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:35.814576 2778 consensus_queue.cc:1035] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "272bc813bd954351bca51bc1223f9898" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:35.814905 2808 consensus_queue.cc:1035] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd02afce634a3d996a7dc2a5631d3b" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:35.821534 2828 mvcc.cc:205] Tried to move back new op lower bound from 7315779894530437120 to 7315779893972758528. Current Snapshot: MvccSnapshot[applied={T|T < 7315779894530437120}]
I20260807 05:04:35.841841 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.837799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:39013 (local address 127.1.17.193:37675)
0807 05:04:35.837954 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.837958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.838255 (+ 297us) server_negotiation.cc:407] Connection header received
0807 05:04:35.838456 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.838459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.838507 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.838586 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:35.838862 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.839382 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.840406 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.840764 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.840864 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.841318 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.841407 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.841461 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.841650 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.841702 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:04:35.841856 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.837724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37675 (local address 127.1.17.195:39013)
0807 05:04:35.838174 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:35.838192 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:35.838354 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:35.838609 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:35.838612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:35.838619 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:04:35.838756 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.838761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.839490 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.839497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.840268 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:35.840281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.841008 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.841012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:35.841105 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.841160 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:04:35.841528 (+ 368us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:04:35.841533 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 05:04:35.841631 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 05:04:35.841709 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":124,"threads_started":1}
I20260807 05:04:35.867169 2245 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:35.867355 2245 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:35.869521 2245 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 248fda7759d5441ea017b21f522e7144: Sending DeleteTablet for 3 replicas of tablet 0faecc1f3c564dfb96320e96fe66cad8
I20260807 05:04:35.870054 2374 tablet_service.cc:1514] Processing DeleteTablet for tablet 0faecc1f3c564dfb96320e96fe66cad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:35 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60642
I20260807 05:04:35.870179 2680 tablet_service.cc:1514] Processing DeleteTablet for tablet 0faecc1f3c564dfb96320e96fe66cad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:35 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:55414
I20260807 05:04:35.870244 2534 tablet_service.cc:1514] Processing DeleteTablet for tablet 0faecc1f3c564dfb96320e96fe66cad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:35 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:39430
I20260807 05:04:35.870497 2850 tablet_replica.cc:330] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: stopping tablet replica
I20260807 05:04:35.870738 2850 raft_consensus.cc:2229] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:35.870914 2850 raft_consensus.cc:2258] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:35.871803 2851 tablet_replica.cc:330] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: stopping tablet replica
I20260807 05:04:35.871804 2850 ts_tablet_manager.cc:1905] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:35.872007 2851 raft_consensus.cc:2229] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:35.872169 2851 raft_consensus.cc:2258] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:35.872875 2851 ts_tablet_manager.cc:1905] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:35.875080 2850 ts_tablet_manager.cc:1918] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:35.875239 2850 log.cc:1196] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/0faecc1f3c564dfb96320e96fe66cad8
I20260807 05:04:35.875890 2851 ts_tablet_manager.cc:1918] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:35.876052 2851 log.cc:1196] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/0faecc1f3c564dfb96320e96fe66cad8
I20260807 05:04:35.876149 2850 ts_tablet_manager.cc:1939] T 0faecc1f3c564dfb96320e96fe66cad8 P 272bc813bd954351bca51bc1223f9898: Deleting consensus metadata
I20260807 05:04:35.876385 2851 ts_tablet_manager.cc:1939] T 0faecc1f3c564dfb96320e96fe66cad8 P 0ffd02afce634a3d996a7dc2a5631d3b: Deleting consensus metadata
I20260807 05:04:35.876972 2229 catalog_manager.cc:4841] TS 272bc813bd954351bca51bc1223f9898 (127.1.17.195:41485): tablet 0faecc1f3c564dfb96320e96fe66cad8 (table test-table [id=eb4f4795c4ab4f479583924a5ae9df38]) successfully deleted
I20260807 05:04:35.877146 2229 catalog_manager.cc:4841] TS 0ffd02afce634a3d996a7dc2a5631d3b (127.1.17.194:40073): tablet 0faecc1f3c564dfb96320e96fe66cad8 (table test-table [id=eb4f4795c4ab4f479583924a5ae9df38]) successfully deleted
I20260807 05:04:35.878222 2849 tablet_replica.cc:330] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: stopping tablet replica
I20260807 05:04:35.878398 2849 raft_consensus.cc:2229] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:35.878734 2849 raft_consensus.cc:2258] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:35.879374 2849 ts_tablet_manager.cc:1905] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:35.882216 2849 ts_tablet_manager.cc:1918] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:35.882339 2849 log.cc:1196] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/0faecc1f3c564dfb96320e96fe66cad8
I20260807 05:04:35.882649 2849 ts_tablet_manager.cc:1939] T 0faecc1f3c564dfb96320e96fe66cad8 P 418e19f83d9e4254b7da2d5cedf1f661: Deleting consensus metadata
I20260807 05:04:35.883308 2232 catalog_manager.cc:4841] TS 418e19f83d9e4254b7da2d5cedf1f661 (127.1.17.193:37675): tablet 0faecc1f3c564dfb96320e96fe66cad8 (table test-table [id=eb4f4795c4ab4f479583924a5ae9df38]) successfully deleted
Aug 07 05:04:35 dist-test-slave-1x4g krb5kdc[2198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:35.889655 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.877590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.1.17.193:37675)
0807 05:04:35.877704 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.877708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.877958 (+ 250us) server_negotiation.cc:407] Connection header received
0807 05:04:35.878069 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.878072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.878117 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.878195 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:35.879124 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.879905 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.881141 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.881475 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.881545 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.885657 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:35.885669 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:35.885684 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:35.885709 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:35.887700 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:35.888188 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:35.888193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:35.888195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:35.888286 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:35.888580 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:35.888583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:35.888585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:35.888921 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:35.889129 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.889337 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.889472 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 05:04:35.890080 2452 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60652
I20260807 05:04:35.902459 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:35.893155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.1.17.254:36663)
0807 05:04:35.893274 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:35.893278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:35.893547 (+ 269us) server_negotiation.cc:407] Connection header received
0807 05:04:35.893637 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:35.893639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:35.893693 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:35.893787 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:35.894557 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.895357 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.896525 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:35.896981 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:35.897086 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:35.898602 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:35.898606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:35.898608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:35.898635 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:35.900523 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:35.901044 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:35.901048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:35.901049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:35.901180 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:35.901490 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:35.901493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:35.901495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:35.901712 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:35.901853 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:35.902102 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 05:04:35.902282 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 05:04:35.902953 2245 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51136
I20260807 05:04:35.904472 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 2323
I20260807 05:04:35.914387 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 2468
I20260807 05:04:35.923833 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 2612
I20260807 05:04:35.931628 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 2211
2026-08-07T05:04:35Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3316 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:35 dist-test-slave-1x4g krb5kdc[2858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:35 dist-test-slave-1x4g krb5kdc[2858](info): set up 2 sockets
Aug 07 05:04:35 dist-test-slave-1x4g krb5kdc[2858](info): commencing operation
krb5kdc: starting...
W20260807 05:04:37.046342 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:37 dist-test-slave-1x4g krb5kdc[2858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:37.169411 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43833
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:37.279058 2872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:37.279325 2872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:37.279362 2872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:37.283104 2872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:37.283145 2872 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 05:04:37.283161 2872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:37.283179 2872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:37.283195 2872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:37.283208 2872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:37.288117 2872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43833
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:37.289026 2872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:37.289951 2872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:37.298106 2878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:37.298115 2877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:37.298148 2880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:37.298380 2872 server_base.cc:1034] running on GCE node
I20260807 05:04:37.298928 2872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:37.299888 2872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:37.301083 2872 hybrid_clock.cc:648] HybridClock initialized: now 1786079077301052 us; error 42 us; skew 500 ppm
Aug 07 05:04:37 dist-test-slave-1x4g krb5kdc[2858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:37.303819 2872 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:37.304677 2872 webserver.cc:460] Webserver started at http://127.1.17.254:36003/ using document root <none> and password file <none>
I20260807 05:04:37.305233 2872 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:37.305295 2872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:37.305457 2872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:37.307346 2872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "21eafe375d514aec8084e156b5ca1749"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "57d3d805515227462d5787a682c7a62b"
server_key_iv: "75c013b5864d4eabc3b7b10650f01f2c"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.307852 2872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "21eafe375d514aec8084e156b5ca1749"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "57d3d805515227462d5787a682c7a62b"
server_key_iv: "75c013b5864d4eabc3b7b10650f01f2c"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.311447 2872 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:04:37.313722 2887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:37.314956 2872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:37.315056 2872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "21eafe375d514aec8084e156b5ca1749"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "57d3d805515227462d5787a682c7a62b"
server_key_iv: "75c013b5864d4eabc3b7b10650f01f2c"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.315137 2872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:37.336138 2872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:37.339290 2872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:37.339438 2872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:37.347596 2872 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:43833
I20260807 05:04:37.347620 2951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:43833 every 8 connection(s)
I20260807 05:04:37.348495 2872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:37.351281 2952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:37.354941 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 2872
I20260807 05:04:37.355053 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:37.355345 1095 external_mini_cluster.cc:1428] Setting key 7df9f22f7b780d6c077dad8ca8ed8c01
I20260807 05:04:37.357323 2952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Bootstrap starting.
I20260807 05:04:37.359436 2952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:37.360184 2952 log.cc:824] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:37.362002 2952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: No bootstrap required, opened a new log
Aug 07 05:04:37 dist-test-slave-1x4g krb5kdc[2858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:37.365688 2952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21eafe375d514aec8084e156b5ca1749" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43833 } }
I20260807 05:04:37.365922 2952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:37.366019 2952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21eafe375d514aec8084e156b5ca1749, State: Initialized, Role: FOLLOWER
I20260807 05:04:37.366451 2952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21eafe375d514aec8084e156b5ca1749" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43833 } }
I20260807 05:04:37.366604 2952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:37.366670 2952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:37.366798 2952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:37.367616 2952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21eafe375d514aec8084e156b5ca1749" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43833 } }
I20260807 05:04:37.367985 2952 leader_election.cc:302] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21eafe375d514aec8084e156b5ca1749; no voters:
I20260807 05:04:37.368309 2952 leader_election.cc:288] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:37.368372 2957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:37.368656 2957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [term 1 LEADER]: Becoming Leader. State: Replica: 21eafe375d514aec8084e156b5ca1749, State: Running, Role: LEADER
I20260807 05:04:37.368968 2957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21eafe375d514aec8084e156b5ca1749" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43833 } }
I20260807 05:04:37.369589 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.356769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.1.17.254:43833)
0807 05:04:37.357205 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:37.357220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:37.357256 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:04:37.357934 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:37.357951 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:37.358258 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:37.358575 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:37.359438 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.360532 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.361448 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.361923 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.362107 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.364563 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:37.364567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:37.364580 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:37.364606 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:37.366501 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.366990 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.367000 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.367005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.367118 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.367434 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.367438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.367440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.367896 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:37.368097 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:37.368386 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 05:04:37.368745 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":127,"threads_started":1}
I20260807 05:04:37.369640 2952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:37.370095 2957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21eafe375d514aec8084e156b5ca1749. Latest consensus state: current_term: 1 leader_uuid: "21eafe375d514aec8084e156b5ca1749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21eafe375d514aec8084e156b5ca1749" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43833 } } }
I20260807 05:04:37.370221 2957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:37.370429 2957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21eafe375d514aec8084e156b5ca1749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21eafe375d514aec8084e156b5ca1749" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43833 } } }
I20260807 05:04:37.370540 2957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749 [sys.catalog]: This master's current role is: LEADER
W20260807 05:04:37.375087 2974 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:04:37.375169 2974 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:04:37.375264 2962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:37.375969 2962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:37.381208 2962 catalog_manager.cc:1347] Generated new cluster ID: dee58a60a0aa45af84b1ab40e030922b
I20260807 05:04:37.381268 2962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:37.420220 2962 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:37.421576 2962 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:37.428506 2962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Generated new TSK 0
I20260807 05:04:37.429325 2962 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:37.458243 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43833
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:37.568674 2979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:37.568945 2979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:37.569022 2979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:37.573189 2979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:37.573239 2979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:37.573303 2979 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 05:04:37.573335 2979 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:04:37.573403 2979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:37.578529 2979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:37.579535 2979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:37.580518 2979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:04:37.590054 2979 server_base.cc:1034] running on GCE node
W20260807 05:04:37.590036 2988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:37.590075 2986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:37.590068 2985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:37.590696 2979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:37.591411 2979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:37.592576 2979 hybrid_clock.cc:648] HybridClock initialized: now 1786079077592563 us; error 23 us; skew 500 ppm
Aug 07 05:04:37 dist-test-slave-1x4g krb5kdc[2858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:37.595504 2979 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:37.596299 2979 webserver.cc:460] Webserver started at http://127.1.17.193:45695/ using document root <none> and password file <none>
I20260807 05:04:37.596947 2979 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:37.597043 2979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:37.597275 2979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:37.599283 2979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "c4ffa30047604cff8d0d9601e523742d"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "b0fdca4436b7be8562d99e58b2c195f9"
server_key_iv: "a0a85dc309100e999569b127302386ed"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.599838 2979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "c4ffa30047604cff8d0d9601e523742d"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "b0fdca4436b7be8562d99e58b2c195f9"
server_key_iv: "a0a85dc309100e999569b127302386ed"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.603618 2979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:04:37.606200 2995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:37.607378 2979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:37.607517 2979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "c4ffa30047604cff8d0d9601e523742d"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "b0fdca4436b7be8562d99e58b2c195f9"
server_key_iv: "a0a85dc309100e999569b127302386ed"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.607630 2979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:37.634361 2979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:37.637753 2979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:37.637975 2979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:37.639405 2979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:37.639477 2979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:37.639547 2979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:37.639596 2979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:04:37 dist-test-slave-1x4g krb5kdc[2858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:37.651041 2979 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:44485
I20260807 05:04:37.651153 3113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:44485 every 8 connection(s)
I20260807 05:04:37.652221 2979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:37.654325 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.640455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:46633 (local address 127.1.17.254:43833)
0807 05:04:37.640596 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:37.640600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:37.641691 (+ 1091us) server_negotiation.cc:407] Connection header received
0807 05:04:37.642992 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:37.642998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:37.643071 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:37.643175 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:37.644889 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.645837 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.647187 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.647547 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.647653 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.650371 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:37.650377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:37.650384 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:37.650426 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:37.652149 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.652844 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.652848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.652850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.652980 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.653375 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.653378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.653379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.653571 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:37.653647 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:37.654096 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 05:04:37.654200 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:04:37.654767 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 2979
I20260807 05:04:37.654896 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:37.655189 3004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.641011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43833 (local address 127.1.17.193:46633)
0807 05:04:37.641529 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:37.641572 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:37.642643 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:37.643540 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:37.643551 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:37.643948 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:37.644701 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.644722 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.645996 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.645999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.647056 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.647065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.647691 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.647695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.647879 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.648483 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:37.648499 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:37.650187 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:37.652288 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.652296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.652313 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.652689 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.653097 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.653100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.653104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.653272 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.653663 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:37.653668 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:37.653880 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 05:04:37.654026 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 05:04:37.654261 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":155,"threads_started":1}
I20260807 05:04:37.655216 1095 external_mini_cluster.cc:1428] Setting key 9ad7e06e1c9d94af48f3b47298ebbfd3
I20260807 05:04:37.656620 3114 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43833
I20260807 05:04:37.656843 3114 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:37.657419 3114 heartbeater.cc:511] Master 127.1.17.254:43833 requested a full tablet report, sending...
I20260807 05:04:37.659003 2904 ts_manager.cc:194] Registered new tserver with Master: c4ffa30047604cff8d0d9601e523742d (127.1.17.193:44485)
I20260807 05:04:37.660486 2904 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:46633
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:04:37.666661 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.658856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:55757 (local address 127.1.17.254:43833)
0807 05:04:37.658956 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:37.658960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:37.659494 (+ 534us) server_negotiation.cc:407] Connection header received
0807 05:04:37.659593 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:37.659596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:37.659635 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:37.659715 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:37.660270 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.660978 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.662063 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.662355 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.662430 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.663674 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:37.663677 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:37.663680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:37.663703 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:37.665111 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.665520 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.665524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.665525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.665605 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.665874 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.665878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.665880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.666046 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:37.666151 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:37.666412 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 05:04:37.666511 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":42}
I20260807 05:04:37.666667 3121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.659056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43833 (local address 127.1.17.193:55757)
0807 05:04:37.659377 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:37.659393 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:37.659513 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:37.659749 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:37.659754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:37.660002 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:37.660142 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.660150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.661135 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.661138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.661925 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.661935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.662583 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.662586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.662692 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.663200 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:37.663211 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:37.663538 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:37.665218 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.665222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.665224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.665422 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.665694 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.665698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.665699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.665786 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.666142 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:37.666146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:37.666235 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 05:04:37.666337 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 05:04:37.666467 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:37.684345 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43833
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:37.795359 3123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:37.795593 3123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:37.795636 3123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:37.800372 3123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:37.800432 3123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:37.800473 3123 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 05:04:37.800522 3123 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:04:37.800585 3123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:37.805989 3123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:37.807047 3123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:37.808090 3123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:37.818280 3128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:37.818338 3129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:37.818295 3131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:37.818675 3123 server_base.cc:1034] running on GCE node
I20260807 05:04:37.819098 3123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:37.819730 3123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:37.820854 3123 hybrid_clock.cc:648] HybridClock initialized: now 1786079077820808 us; error 56 us; skew 500 ppm
Aug 07 05:04:37 dist-test-slave-1x4g krb5kdc[2858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:37.823557 3123 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:37.824460 3123 webserver.cc:460] Webserver started at http://127.1.17.194:34985/ using document root <none> and password file <none>
I20260807 05:04:37.825090 3123 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:37.825155 3123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:37.825403 3123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:37.828181 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "a8370a8803c44149814f04be9622e45d"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "a49407a4417e748dcb3db61f1de72464"
server_key_iv: "95609af36c2be30efe077bb7989a2311"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.828840 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "a8370a8803c44149814f04be9622e45d"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "a49407a4417e748dcb3db61f1de72464"
server_key_iv: "95609af36c2be30efe077bb7989a2311"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.832682 3123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:04:37.835137 3139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:37.836370 3123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:37.836516 3123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "a8370a8803c44149814f04be9622e45d"
format_stamp: "Formatted at 2026-08-07 05:04:37 on dist-test-slave-1x4g"
server_key: "a49407a4417e748dcb3db61f1de72464"
server_key_iv: "95609af36c2be30efe077bb7989a2311"
server_key_version: "encryptionkey@0"
I20260807 05:04:37.836637 3123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:37.853466 3123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:37.856297 3123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:37.856498 3123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:37.858158 3123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:37.858228 3123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:37.858296 3123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:37.858330 3123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:04:37 dist-test-slave-1x4g krb5kdc[2858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:37.869164 3123 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:41961
I20260807 05:04:37.869274 3258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:41961 every 8 connection(s)
I20260807 05:04:37.870177 3123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:37.870975 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 3123
I20260807 05:04:37.871105 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:37.871434 1095 external_mini_cluster.cc:1428] Setting key 8ebe2d8e6b545ea7e1179c3537cd0e4e
I20260807 05:04:37.873430 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.858853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:56569 (local address 127.1.17.254:43833)
0807 05:04:37.859023 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:37.859039 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:37.860377 (+ 1338us) server_negotiation.cc:407] Connection header received
0807 05:04:37.861859 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:37.861863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:37.861920 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:37.862015 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:37.863261 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.863996 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.865387 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.865639 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.865754 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.868942 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:37.868948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:37.868953 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:37.868985 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:37.870652 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.871699 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.871703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.871705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.871804 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.872240 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.872245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.872250 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.872498 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:37.872580 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:37.873201 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 05:04:37.873314 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:04:37.874500 3149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.859753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43833 (local address 127.1.17.194:56569)
0807 05:04:37.860225 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:37.860277 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:37.861640 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:37.862230 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:37.862238 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:37.862626 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:37.863101 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.863114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.864129 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.864132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.865253 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.865263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.865806 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.865810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.865992 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.867146 (+ 1154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:37.867166 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:37.868733 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:37.871206 (+ 2473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.871211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.871261 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.871566 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.871927 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.871930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.871931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.872128 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.872604 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:37.872610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:37.872913 (+ 303us) client_negotiation.cc:769] Sending connection context
0807 05:04:37.873092 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 05:04:37.873434 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":144,"threads_started":1}
I20260807 05:04:37.876819 3259 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43833
I20260807 05:04:37.876967 3259 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:37.877511 3259 heartbeater.cc:511] Master 127.1.17.254:43833 requested a full tablet report, sending...
I20260807 05:04:37.878569 2903 ts_manager.cc:194] Registered new tserver with Master: a8370a8803c44149814f04be9622e45d (127.1.17.194:41961)
I20260807 05:04:37.879184 2903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:56569
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
I20260807 05:04:37.888051 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.878102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:45173 (local address 127.1.17.254:43833)
0807 05:04:37.878215 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:37.878219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:37.881047 (+ 2828us) server_negotiation.cc:407] Connection header received
0807 05:04:37.881180 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:37.881183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:37.881228 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:37.881309 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:37.881763 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.882257 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.883110 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.883316 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.883421 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.884512 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:37.884515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:37.884518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:37.884547 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:37.886256 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.886688 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.886694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.886699 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.886794 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:37.887056 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:37.887062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:37.887066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:37.887388 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:37.887498 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:37.887753 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 05:04:37.887857 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:04:37.888058 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:37.878025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43833 (local address 127.1.17.194:45173)
0807 05:04:37.880961 (+ 2936us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:37.880974 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:37.881087 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:37.881327 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:37.881330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:37.881550 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:37.881644 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.881650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.882380 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.882383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.882999 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:37.883007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:37.883434 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:37.883438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:37.883545 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:37.884043 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:37.884054 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:37.884366 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:37.886375 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.886377 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.886379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.886590 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.886889 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:37.886891 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:37.886892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:37.886969 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:37.887492 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:37.887495 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:37.887592 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 05:04:37.887698 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 05:04:37.887840 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2851,"thread_start_us":276,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:37.901911 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43833
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:38.015625 3268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:38.015852 3268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:38.015924 3268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:38.019972 3268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:38.020022 3268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:38.020098 3268 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 05:04:38.020129 3268 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:04:38.020208 3268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:38.025373 3268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:43833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:38.026347 3268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:38.027329 3268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:38.036777 3276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:38.036870 3268 server_base.cc:1034] running on GCE node
W20260807 05:04:38.036836 3273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:38.036791 3274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:38.037514 3268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:38.038154 3268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:38.039350 3268 hybrid_clock.cc:648] HybridClock initialized: now 1786079078039337 us; error 28 us; skew 500 ppm
Aug 07 05:04:38 dist-test-slave-1x4g krb5kdc[2858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:38.042178 3268 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:38.043053 3268 webserver.cc:460] Webserver started at http://127.1.17.195:39077/ using document root <none> and password file <none>
I20260807 05:04:38.043618 3268 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:38.043711 3268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:38.043944 3268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:38.046011 3268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26"
format_stamp: "Formatted at 2026-08-07 05:04:38 on dist-test-slave-1x4g"
server_key: "c62b8fd4dd3c84f6e70d8a5293a31fbe"
server_key_iv: "5a922b5f7868a77a6ae922a6d264003a"
server_key_version: "encryptionkey@0"
I20260807 05:04:38.046593 3268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26"
format_stamp: "Formatted at 2026-08-07 05:04:38 on dist-test-slave-1x4g"
server_key: "c62b8fd4dd3c84f6e70d8a5293a31fbe"
server_key_iv: "5a922b5f7868a77a6ae922a6d264003a"
server_key_version: "encryptionkey@0"
I20260807 05:04:38.050382 3268 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:04:38.052907 3284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:38.054171 3268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:38.054311 3268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26"
format_stamp: "Formatted at 2026-08-07 05:04:38 on dist-test-slave-1x4g"
server_key: "c62b8fd4dd3c84f6e70d8a5293a31fbe"
server_key_iv: "5a922b5f7868a77a6ae922a6d264003a"
server_key_version: "encryptionkey@0"
I20260807 05:04:38.054419 3268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:38.067380 3268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:38.070766 3268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:38.070955 3268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:38.072546 3268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:38.072607 3268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:38.072656 3268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:38.072695 3268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:04:38 dist-test-slave-1x4g krb5kdc[2858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:38.083667 3268 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:44155
I20260807 05:04:38.083755 3400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:44155 every 8 connection(s)
I20260807 05:04:38.084822 3268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:38.087330 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.073312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:33907 (local address 127.1.17.254:43833)
0807 05:04:38.073481 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.073487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.074794 (+ 1307us) server_negotiation.cc:407] Connection header received
0807 05:04:38.076225 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.076228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.076282 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.076394 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:38.078036 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.078777 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.080192 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.080540 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.080668 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.083406 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:38.083411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:38.083415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:38.083453 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:38.085247 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.085900 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.085904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.085905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.086002 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.086342 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.086344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.086345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.086522 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:38.086613 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.087107 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.087207 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 05:04:38.088157 3293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.074113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43833 (local address 127.1.17.195:33907)
0807 05:04:38.074609 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.074673 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.075957 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.076593 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.076617 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.077156 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:38.077880 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.077908 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.078936 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.078942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.080042 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.080055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.080725 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.080729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.080968 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.081571 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:38.081617 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:38.083207 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:38.085396 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:38.085402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:38.085416 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:38.085774 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:38.086104 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:38.086106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:38.086108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:38.086253 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:38.086630 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:38.086634 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:38.086886 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.087060 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.087302 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260807 05:04:38.089100 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 3268
I20260807 05:04:38.089254 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:38.089542 3401 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43833
I20260807 05:04:38.089614 1095 external_mini_cluster.cc:1428] Setting key ec01a5fef716aedccd27a078b9893594
I20260807 05:04:38.089704 3401 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:38.090313 3401 heartbeater.cc:511] Master 127.1.17.254:43833 requested a full tablet report, sending...
I20260807 05:04:38.091621 2903 ts_manager.cc:194] Registered new tserver with Master: fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:38.092442 2903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:33907
I20260807 05:04:38.099098 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.092115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:60639 (local address 127.1.17.254:43833)
0807 05:04:38.092232 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.092236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.092292 (+ 56us) server_negotiation.cc:407] Connection header received
0807 05:04:38.092422 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.092424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.092456 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.092547 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:38.093090 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.093606 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.094475 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.094714 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.094834 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.095989 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:38.095994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:38.095996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:38.096023 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:38.097338 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.097803 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.097809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.097811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.097906 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.098220 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.098223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.098225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.098423 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:38.098538 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.098834 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.098942 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:04:38.099119 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.091794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43833 (local address 127.1.17.195:60639)
0807 05:04:38.092221 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.092236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.092335 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.092565 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.092568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.092812 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:38.092959 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.092966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.093746 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.093750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.094341 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.094351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.094836 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.094840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.094963 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.095408 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:38.095429 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:38.095775 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:38.097467 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:38.097473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:38.097475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:38.097688 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:38.098010 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:38.098015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:38.098017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:38.098108 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:38.098539 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:38.098543 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:38.098644 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.098775 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.098936 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":103,"threads_started":1}
I20260807 05:04:38.104288 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:04:38 dist-test-slave-1x4g krb5kdc[2858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:04:38 dist-test-slave-1x4g krb5kdc[2858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:38.131273 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.122991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.1.17.254:43833)
0807 05:04:38.123126 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.123130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.123142 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:04:38.123180 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.123182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.123218 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.123309 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:38.124055 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.124571 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.125514 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.125725 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.125833 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.128171 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:38.128174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:38.128176 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:38.128199 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:38.129634 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.130054 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.130058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.130059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.130142 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.130416 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.130419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.130421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.130599 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:38.130730 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.130976 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.131106 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:04:38.134171 2903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:04:38.137279 2903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:38.151176 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.145558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54360 (local address 127.1.17.194:41961)
0807 05:04:38.146969 (+ 1411us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.146980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.146998 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:04:38.147066 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.147070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.147218 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.147556 (+ 338us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:38.147700 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.148397 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.149807 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.150472 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.150606 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.150775 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.150934 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.151024 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1304,"thread_start_us":109,"threads_started":1}
I20260807 05:04:38.151165 3419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.145348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:41961 (local address 127.0.0.1:54360)
0807 05:04:38.145701 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.145892 (+ 191us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.146022 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.147362 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.147365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.147391 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:38.147581 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.147590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.148511 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.148517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.149686 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.149695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.150615 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.150619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.150727 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.150769 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.150901 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.151020 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":48,"thread_start_us":94,"threads_started":1}
I20260807 05:04:38.151494 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.145683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44485 (local address 127.0.0.1:60720)
0807 05:04:38.146341 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.146353 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.146444 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.147175 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.147181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.147204 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:38.147356 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.147363 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.148792 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.148796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.149985 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.149995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.151115 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.151118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.151226 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.151247 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.151299 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.151374 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260807 05:04:38.152163 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.146097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:44155 (local address 127.0.0.1:47490)
0807 05:04:38.146602 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.146612 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.146706 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.147355 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.147358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.147373 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:38.147525 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.147544 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.148979 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.148982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.150061 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.150068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.151115 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.151118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.151830 (+ 712us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.151859 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.151960 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.152037 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":607,"spinlock_wait_cycles":6912,"thread_start_us":95,"threads_started":1}
I20260807 05:04:38.152448 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.146512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47490 (local address 127.1.17.195:44155)
0807 05:04:38.146868 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.146880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.146902 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:04:38.146980 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.146986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.147185 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.147394 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:38.147665 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.148866 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.150192 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.150967 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.151162 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.151320 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.151994 (+ 674us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.152080 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":125,"threads_started":1}
I20260807 05:04:38.152936 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.146190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.1.17.193:44485)
0807 05:04:38.146740 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.146756 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.146782 (+ 26us) server_negotiation.cc:407] Connection header received
0807 05:04:38.146864 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.146868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.147024 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.147190 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:38.147495 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.148648 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.150161 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.150967 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.152451 (+ 1484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.152628 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.152679 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.152770 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":74,"threads_started":1}
I20260807 05:04:38.154533 3191 tablet_service.cc:1467] Processing CreateTablet for tablet eaacd35cc269468eb2964cb9be21f71b (DEFAULT_TABLE table=test-table [id=b71b4d2d704342ab957f1c2c8bf844cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:38.154865 3335 tablet_service.cc:1467] Processing CreateTablet for tablet eaacd35cc269468eb2964cb9be21f71b (DEFAULT_TABLE table=test-table [id=b71b4d2d704342ab957f1c2c8bf844cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:38.155681 3191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaacd35cc269468eb2964cb9be21f71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.155867 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaacd35cc269468eb2964cb9be21f71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.155539 3045 tablet_service.cc:1467] Processing CreateTablet for tablet eaacd35cc269468eb2964cb9be21f71b (DEFAULT_TABLE table=test-table [id=b71b4d2d704342ab957f1c2c8bf844cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:38.156454 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaacd35cc269468eb2964cb9be21f71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.162396 3425 tablet_bootstrap.cc:492] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Bootstrap starting.
I20260807 05:04:38.164160 3426 tablet_bootstrap.cc:492] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Bootstrap starting.
I20260807 05:04:38.164708 3425 tablet_bootstrap.cc:654] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.165447 3425 log.cc:824] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:38.166049 3427 tablet_bootstrap.cc:492] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Bootstrap starting.
I20260807 05:04:38.166361 3426 tablet_bootstrap.cc:654] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.167126 3426 log.cc:824] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:38.167470 3425 tablet_bootstrap.cc:492] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: No bootstrap required, opened a new log
I20260807 05:04:38.167661 3425 ts_tablet_manager.cc:1397] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:04:38.167815 3427 tablet_bootstrap.cc:654] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.168721 3427 log.cc:824] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:38.169154 3426 tablet_bootstrap.cc:492] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: No bootstrap required, opened a new log
I20260807 05:04:38.169345 3426 ts_tablet_manager.cc:1397] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:04:38.170686 3427 tablet_bootstrap.cc:492] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: No bootstrap required, opened a new log
I20260807 05:04:38.170943 3427 ts_tablet_manager.cc:1397] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:04:38.171720 3425 raft_consensus.cc:348] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.171993 3425 raft_consensus.cc:374] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.172057 3425 raft_consensus.cc:729] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9c1ef92ac14abaa5b394d9d0e7eb26, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.172487 3425 consensus_queue.cc:260] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.173209 3401 heartbeater.cc:503] Master 127.1.17.254:43833 was elected leader, sending a full tablet report...
I20260807 05:04:38.173208 3426 raft_consensus.cc:348] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.173442 3426 raft_consensus.cc:374] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.173491 3426 raft_consensus.cc:729] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.173570 3425 ts_tablet_manager.cc:1428] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260807 05:04:38.173933 3426 consensus_queue.cc:260] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.174590 3114 heartbeater.cc:503] Master 127.1.17.254:43833 was elected leader, sending a full tablet report...
I20260807 05:04:38.174813 3426 ts_tablet_manager.cc:1428] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:04:38.175680 3427 raft_consensus.cc:348] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.175928 3427 raft_consensus.cc:374] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.176005 3427 raft_consensus.cc:729] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8370a8803c44149814f04be9622e45d, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.176622 3427 consensus_queue.cc:260] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.177597 3427 ts_tablet_manager.cc:1428] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260807 05:04:38.178018 3259 heartbeater.cc:503] Master 127.1.17.254:43833 was elected leader, sending a full tablet report...
I20260807 05:04:38.187222 3432 raft_consensus.cc:482] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:38.187345 3432 raft_consensus.cc:504] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.188150 3432 leader_election.cc:288] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:38.192426 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.188459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:49679 (local address 127.1.17.194:41961)
0807 05:04:38.188579 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.188583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.188747 (+ 164us) server_negotiation.cc:407] Connection header received
0807 05:04:38.188927 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.188933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.188990 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.189079 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:38.189378 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.190210 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.191412 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.191895 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.191978 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.192013 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.192238 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.192307 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:04:38.192453 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.188379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:41961 (local address 127.1.17.193:49679)
0807 05:04:38.188633 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.188650 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.188758 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.189084 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.189086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.189103 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:38.189202 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.189207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.190315 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.190319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.191265 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.191274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.191997 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.192000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.192090 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.192114 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.192203 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.192269 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260807 05:04:38.192682 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.188798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:44155 (local address 127.1.17.193:51821)
0807 05:04:38.189131 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.189144 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.189235 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.189639 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.189642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.189658 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:38.189765 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.189771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.190686 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.190689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.191547 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.191558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.192251 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.192254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.192348 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.192370 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.192449 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.192516 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":25,"thread_start_us":125,"threads_started":1}
I20260807 05:04:38.192687 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.188882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:51821 (local address 127.1.17.195:44155)
0807 05:04:38.189008 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.189012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.189205 (+ 193us) server_negotiation.cc:407] Connection header received
0807 05:04:38.189386 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.189389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.189459 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.189507 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:38.189856 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.190521 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.191665 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.192099 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.192229 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.192270 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.192479 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.192561 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:04:38.193078 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaacd35cc269468eb2964cb9be21f71b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d" is_pre_election: true
I20260807 05:04:38.193337 3211 raft_consensus.cc:2454] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:38.193801 2999 leader_election.cc:302] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8370a8803c44149814f04be9622e45d, c4ffa30047604cff8d0d9601e523742d; no voters:
I20260807 05:04:38.194089 3432 raft_consensus.cc:2781] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:38.194083 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaacd35cc269468eb2964cb9be21f71b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" is_pre_election: true
I20260807 05:04:38.194185 3432 raft_consensus.cc:482] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:38.194235 3432 raft_consensus.cc:3037] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.194339 3354 raft_consensus.cc:2454] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:38.195284 3432 raft_consensus.cc:504] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.195595 3432 leader_election.cc:288] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Requested vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:38.195925 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaacd35cc269468eb2964cb9be21f71b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d"
I20260807 05:04:38.196007 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaacd35cc269468eb2964cb9be21f71b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26"
I20260807 05:04:38.196059 3211 raft_consensus.cc:3037] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.196131 3354 raft_consensus.cc:3037] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.197147 3211 raft_consensus.cc:2454] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:38.197218 3354 raft_consensus.cc:2454] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:38.197469 2999 leader_election.cc:302] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8370a8803c44149814f04be9622e45d, c4ffa30047604cff8d0d9601e523742d; no voters:
I20260807 05:04:38.197629 3432 raft_consensus.cc:2781] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:38.197861 3432 raft_consensus.cc:686] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Becoming Leader. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Running, Role: LEADER
I20260807 05:04:38.198176 3432 consensus_queue.cc:237] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } }
I20260807 05:04:38.201138 2903 catalog_manager.cc:5495] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d reported cstate change: term changed from 0 to 1, leader changed from <none> to c4ffa30047604cff8d0d9601e523742d (127.1.17.193). New cstate: current_term: 1 leader_uuid: "c4ffa30047604cff8d0d9601e523742d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } health_report { overall_health: HEALTHY } } }
Aug 07 05:04:38 dist-test-slave-1x4g krb5kdc[2858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:38.237990 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.230237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43833 (local address 127.0.0.1:50806)
0807 05:04:38.230504 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.230517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.230607 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.230813 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.230815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.231191 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:38.231289 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.231294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.231985 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.231987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.232517 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.232524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.232925 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.232928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.233032 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.233599 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:38.233611 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:38.234989 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:38.236628 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:38.236635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:38.236640 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:38.236911 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:38.237154 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:38.237155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:38.237157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:38.237233 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:38.237513 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:38.237518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:38.237611 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.237702 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.237815 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260807 05:04:38.237998 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.230366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.1.17.254:43833)
0807 05:04:38.230507 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.230511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.230584 (+ 73us) server_negotiation.cc:407] Connection header received
0807 05:04:38.230678 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.230680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.230715 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.230778 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:38.231399 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.231881 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.232611 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.232821 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.232891 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.235110 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:38.235113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:38.235115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:38.235136 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:38.236508 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.236984 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.236986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.236988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.237074 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:38.237289 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:38.237291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:38.237292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:38.237432 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:38.237500 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.237748 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.237883 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":56}
I20260807 05:04:38.241766 2903 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:50806:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 05:04:38.242493 2903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:38.247148 3045 tablet_service.cc:1467] Processing CreateTablet for tablet bf0494e761574231beb825898b9126d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0df4ea8e36794cbbb903f449a69838f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:04:38.247424 3335 tablet_service.cc:1467] Processing CreateTablet for tablet bf0494e761574231beb825898b9126d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0df4ea8e36794cbbb903f449a69838f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:04:38.247394 3191 tablet_service.cc:1467] Processing CreateTablet for tablet bf0494e761574231beb825898b9126d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0df4ea8e36794cbbb903f449a69838f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:04:38.247848 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0494e761574231beb825898b9126d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.248076 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0494e761574231beb825898b9126d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.248080 3191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf0494e761574231beb825898b9126d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.250450 3426 tablet_bootstrap.cc:492] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: Bootstrap starting.
I20260807 05:04:38.250916 3427 tablet_bootstrap.cc:492] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: Bootstrap starting.
I20260807 05:04:38.251133 3425 tablet_bootstrap.cc:492] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Bootstrap starting.
I20260807 05:04:38.251464 3426 tablet_bootstrap.cc:654] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.251865 3427 tablet_bootstrap.cc:654] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.252092 3425 tablet_bootstrap.cc:654] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.253011 3426 tablet_bootstrap.cc:492] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: No bootstrap required, opened a new log
I20260807 05:04:38.253106 3426 ts_tablet_manager.cc:1397] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:38.253325 3427 tablet_bootstrap.cc:492] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: No bootstrap required, opened a new log
I20260807 05:04:38.253407 3427 ts_tablet_manager.cc:1397] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 05:04:38.253630 3426 raft_consensus.cc:348] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.253717 3426 raft_consensus.cc:374] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.253739 3426 raft_consensus.cc:729] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.253793 3425 tablet_bootstrap.cc:492] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: No bootstrap required, opened a new log
I20260807 05:04:38.253870 3425 ts_tablet_manager.cc:1397] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 05:04:38.253862 3426 consensus_queue.cc:260] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.253876 3427 raft_consensus.cc:348] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.253983 3427 raft_consensus.cc:374] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.254021 3427 raft_consensus.cc:729] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8370a8803c44149814f04be9622e45d, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.254119 3432 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.254158 3427 consensus_queue.cc:260] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.254256 3432 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:38.254129 3426 ts_tablet_manager.cc:1428] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:04:38.254334 3425 raft_consensus.cc:348] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.254437 3425 raft_consensus.cc:374] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.254474 3425 raft_consensus.cc:729] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9c1ef92ac14abaa5b394d9d0e7eb26, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.254437 3433 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.254570 3433 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:38.254606 3425 consensus_queue.cc:260] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.254806 3427 ts_tablet_manager.cc:1428] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:38.254876 3425 ts_tablet_manager.cc:1428] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:04:38.255064 3431 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.255177 3431 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 05:04:38.336293 3402 tablet.cc:2369] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:04:38.356824 3431 raft_consensus.cc:482] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:38.356973 3431 raft_consensus.cc:504] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.358247 3431 leader_election.cc:288] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4ffa30047604cff8d0d9601e523742d (127.1.17.193:44485), a8370a8803c44149814f04be9622e45d (127.1.17.194:41961)
I20260807 05:04:38.362412 3293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.358441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44485 (local address 127.1.17.195:45081)
0807 05:04:38.358713 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.358727 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.358807 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.359221 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.359223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.359236 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:38.359334 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.359338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.360208 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.360211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.361265 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.361275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.362032 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.362036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.362130 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.362164 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.362235 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.362304 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260807 05:04:38.362485 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.358884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:45081 (local address 127.1.17.193:44485)
0807 05:04:38.359009 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.359013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.359027 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:04:38.359077 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.359080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.359127 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.359196 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:38.359452 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.360085 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.361396 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.361900 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.361994 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.362055 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.362269 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.362350 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:04:38.362741 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.358610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:50711 (local address 127.1.17.194:41961)
0807 05:04:38.358741 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.358746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.359290 (+ 544us) server_negotiation.cc:407] Connection header received
0807 05:04:38.359347 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.359350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.359392 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.359462 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:38.359841 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.360429 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.361644 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.362119 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.362248 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.362293 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.362506 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.362604 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:04:38.362972 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.358477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:41961 (local address 127.1.17.195:50711)
0807 05:04:38.358853 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.358866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.359014 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.359563 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.359566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.359581 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:38.359706 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.359711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.360560 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.360564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.361491 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.361501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.362276 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.362279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.362365 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.362386 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.362458 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.362827 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":109,"threads_started":1}
I20260807 05:04:38.363032 3067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0494e761574231beb825898b9126d0" candidate_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ffa30047604cff8d0d9601e523742d" is_pre_election: true
I20260807 05:04:38.363289 3067 raft_consensus.cc:2454] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc9c1ef92ac14abaa5b394d9d0e7eb26 in term 0.
I20260807 05:04:38.363444 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0494e761574231beb825898b9126d0" candidate_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d" is_pre_election: true
I20260807 05:04:38.363639 3211 raft_consensus.cc:2454] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc9c1ef92ac14abaa5b394d9d0e7eb26 in term 0.
I20260807 05:04:38.363829 3286 leader_election.cc:302] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4ffa30047604cff8d0d9601e523742d, fc9c1ef92ac14abaa5b394d9d0e7eb26; no voters:
I20260807 05:04:38.364152 3431 raft_consensus.cc:2781] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:38.364221 3431 raft_consensus.cc:482] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:38.364249 3431 raft_consensus.cc:3037] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.365280 3431 raft_consensus.cc:504] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.365681 3431 leader_election.cc:288] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [CANDIDATE]: Term 1 election: Requested vote from peers c4ffa30047604cff8d0d9601e523742d (127.1.17.193:44485), a8370a8803c44149814f04be9622e45d (127.1.17.194:41961)
I20260807 05:04:38.366039 3067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0494e761574231beb825898b9126d0" candidate_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4ffa30047604cff8d0d9601e523742d"
I20260807 05:04:38.366075 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf0494e761574231beb825898b9126d0" candidate_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d"
I20260807 05:04:38.366182 3067 raft_consensus.cc:3037] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.366197 3211 raft_consensus.cc:3037] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.367086 3211 raft_consensus.cc:2454] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc9c1ef92ac14abaa5b394d9d0e7eb26 in term 1.
I20260807 05:04:38.367201 3067 raft_consensus.cc:2454] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc9c1ef92ac14abaa5b394d9d0e7eb26 in term 1.
I20260807 05:04:38.367373 3288 leader_election.cc:302] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8370a8803c44149814f04be9622e45d, fc9c1ef92ac14abaa5b394d9d0e7eb26; no voters:
I20260807 05:04:38.367580 3431 raft_consensus.cc:2781] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:38.367781 3431 raft_consensus.cc:686] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 LEADER]: Becoming Leader. State: Replica: fc9c1ef92ac14abaa5b394d9d0e7eb26, State: Running, Role: LEADER
I20260807 05:04:38.368100 3431 consensus_queue.cc:237] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.370476 3448 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: TxnStatusTablet state changed. Reason: New leader fc9c1ef92ac14abaa5b394d9d0e7eb26. Latest consensus state: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.370582 3448 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:04:38.371068 3450 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:04:38.371276 3450 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260807 05:04:38.371461 3260 tablet.cc:2369] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:04:38.371351 2904 catalog_manager.cc:5495] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195). New cstate: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } health_report { overall_health: HEALTHY } } }
W20260807 05:04:38.404062 3115 tablet.cc:2369] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:04:38.410959 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.406816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:44155 (local address 127.0.0.1:47500)
0807 05:04:38.406969 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.406986 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.407118 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.407468 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.407471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.407476 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:04:38.407579 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.407583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.408559 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.408561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.409200 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.409209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.409788 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.409791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.409859 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.409901 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:04:38.410666 (+ 765us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:04:38.410679 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.410782 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.410851 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 05:04:38.410984 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.406944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.1.17.195:44155)
0807 05:04:38.407072 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.407076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.407091 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:04:38.407270 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.407274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.407337 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.407454 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:38.407684 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.408451 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.409311 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.409682 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.409790 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.410038 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:38.410538 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:38.410673 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.410802 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.410856 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:04:38.416036 3067 raft_consensus.cc:1261] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Refusing update from remote peer fc9c1ef92ac14abaa5b394d9d0e7eb26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:38.416045 3211 raft_consensus.cc:1261] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Refusing update from remote peer fc9c1ef92ac14abaa5b394d9d0e7eb26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:38.416706 3448 consensus_queue.cc:1035] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:38.416986 3431 consensus_queue.cc:1035] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:38.420830 3432 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: TxnStatusTablet state changed. Reason: New leader fc9c1ef92ac14abaa5b394d9d0e7eb26. Latest consensus state: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.420951 3432 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:38.422266 3433 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: TxnStatusTablet state changed. Reason: New leader fc9c1ef92ac14abaa5b394d9d0e7eb26. Latest consensus state: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.422399 3433 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:38.422742 3454 mvcc.cc:205] Tried to move back new op lower bound from 7315779905185394688 to 7315779904998359040. Current Snapshot: MvccSnapshot[applied={T|T < 7315779905185394688}]
I20260807 05:04:38.423285 3431 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.423395 3431 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:04:38.424639 3432 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.424763 3432 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P c4ffa30047604cff8d0d9601e523742d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:38.425025 3456 mvcc.cc:205] Tried to move back new op lower bound from 7315779905185394688 to 7315779904998359040. Current Snapshot: MvccSnapshot[applied={T|T < 7315779905185394688}]
I20260807 05:04:38.429220 3431 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.429361 3431 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P fc9c1ef92ac14abaa5b394d9d0e7eb26: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:04:38.430716 3433 tablet_replica.cc:439] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } } }
I20260807 05:04:38.430835 3433 tablet_replica.cc:442] T bf0494e761574231beb825898b9126d0 P a8370a8803c44149814f04be9622e45d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:04:38.436193 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.432540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60732 (local address 127.1.17.193:44485)
0807 05:04:38.432653 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.432663 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.432675 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:04:38.432860 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.432863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.432918 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.433037 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:38.433307 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.433840 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.434805 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.435043 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.435151 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.435438 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:38.435803 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:38.435932 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.435997 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.436056 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:04:38.446545 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.443113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:50243 (local address 127.1.17.195:44155)
0807 05:04:38.443225 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.443229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.443366 (+ 137us) server_negotiation.cc:407] Connection header received
0807 05:04:38.443496 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.443498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.443544 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.443617 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:38.443886 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.444350 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.445613 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.445807 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.445875 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.446189 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:38.446246 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:38.446298 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.446418 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.446458 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:04:38.446622 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.442977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:44155 (local address 127.1.17.193:50243)
0807 05:04:38.443282 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.443296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.443398 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.443650 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.443653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.443660 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:04:38.443788 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.443793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.444454 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.444458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.445496 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.445507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.445906 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.445910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.446001 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.446046 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:04:38.446325 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:04:38.446331 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.446402 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.446472 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":119,"threads_started":1}
I20260807 05:04:38.453133 3211 raft_consensus.cc:1261] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:38.453433 3354 raft_consensus.cc:1261] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:38.453679 3432 consensus_queue.cc:1035] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:38.454252 3432 consensus_queue.cc:1035] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:38.458925 3455 mvcc.cc:205] Tried to move back new op lower bound from 7315779905339809792 to 7315779904302260224. Current Snapshot: MvccSnapshot[applied={T|T < 7315779905339809792}]
I20260807 05:04:38.478268 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.474297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44485 (local address 127.1.17.195:43161)
0807 05:04:38.474581 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:38.474603 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:38.474713 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:38.474965 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:38.474967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:38.474974 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:04:38.475068 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.475076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.476197 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.476201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.476972 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:38.476982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.477559 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.477561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:38.477621 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.477665 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:04:38.477966 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:04:38.477972 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:04:38.478037 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 05:04:38.478128 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260807 05:04:38.478271 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:38.474411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:43161 (local address 127.1.17.193:44485)
0807 05:04:38.474515 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:38.474520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:38.474665 (+ 145us) server_negotiation.cc:407] Connection header received
0807 05:04:38.474823 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:38.474827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:38.474872 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:38.474955 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:38.475174 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.475938 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.477088 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:38.477435 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:38.477541 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:38.477799 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:38.477882 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:38.477960 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:38.478090 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 05:04:38.478146 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 05:04:38.505802 2903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:38.506021 2903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:38.508920 2903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Sending DeleteTablet for 3 replicas of tablet eaacd35cc269468eb2964cb9be21f71b
I20260807 05:04:38.509902 3045 tablet_service.cc:1514] Processing DeleteTablet for tablet eaacd35cc269468eb2964cb9be21f71b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:38 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60720
I20260807 05:04:38.509914 3191 tablet_service.cc:1514] Processing DeleteTablet for tablet eaacd35cc269468eb2964cb9be21f71b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:38 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:54360
I20260807 05:04:38.510643 3474 tablet_replica.cc:330] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: stopping tablet replica
I20260807 05:04:38.510999 3474 raft_consensus.cc:2229] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:38.511113 2903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:04:38.511503 2903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:38.511790 3474 raft_consensus.cc:2258] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:38.512217 3335 tablet_service.cc:1514] Processing DeleteTablet for tablet eaacd35cc269468eb2964cb9be21f71b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:38 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:47490
I20260807 05:04:38.513055 3476 tablet_replica.cc:330] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: stopping tablet replica
I20260807 05:04:38.513346 3476 raft_consensus.cc:2229] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:38.513528 3476 raft_consensus.cc:2258] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:38.513844 3475 tablet_replica.cc:330] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: stopping tablet replica
I20260807 05:04:38.514125 3475 raft_consensus.cc:2229] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:38.514328 3475 raft_consensus.cc:2258] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:38.516203 3474 ts_tablet_manager.cc:1905] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:38.517464 3335 tablet_service.cc:1467] Processing CreateTablet for tablet 899b552a65f4491a8c04db78afd9ed0b (DEFAULT_TABLE table=test-table [id=ddf36c48bfcd420988fe92d6ea005f99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:38.517504 3191 tablet_service.cc:1467] Processing CreateTablet for tablet 899b552a65f4491a8c04db78afd9ed0b (DEFAULT_TABLE table=test-table [id=ddf36c48bfcd420988fe92d6ea005f99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:38.517753 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 899b552a65f4491a8c04db78afd9ed0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.517776 3191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 899b552a65f4491a8c04db78afd9ed0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.516203 3476 ts_tablet_manager.cc:1905] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:38.520345 3427 tablet_bootstrap.cc:492] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: Bootstrap starting.
I20260807 05:04:38.520371 3425 tablet_bootstrap.cc:492] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Bootstrap starting.
I20260807 05:04:38.521287 3427 tablet_bootstrap.cc:654] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.521565 3425 tablet_bootstrap.cc:654] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.522747 3045 tablet_service.cc:1467] Processing CreateTablet for tablet 899b552a65f4491a8c04db78afd9ed0b (DEFAULT_TABLE table=test-table [id=ddf36c48bfcd420988fe92d6ea005f99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:38.523003 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 899b552a65f4491a8c04db78afd9ed0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:38.523511 3427 tablet_bootstrap.cc:492] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: No bootstrap required, opened a new log
I20260807 05:04:38.523594 3427 ts_tablet_manager.cc:1397] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 05:04:38.523996 3427 raft_consensus.cc:348] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.524101 3427 raft_consensus.cc:374] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.524132 3427 raft_consensus.cc:729] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8370a8803c44149814f04be9622e45d, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.524243 3427 consensus_queue.cc:260] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.524474 3427 ts_tablet_manager.cc:1428] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:04:38.525306 3426 tablet_bootstrap.cc:492] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: Bootstrap starting.
I20260807 05:04:38.526202 3426 tablet_bootstrap.cc:654] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:38.528252 3426 tablet_bootstrap.cc:492] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: No bootstrap required, opened a new log
I20260807 05:04:38.528329 3426 ts_tablet_manager.cc:1397] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:38.528928 3426 raft_consensus.cc:348] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.529028 3426 raft_consensus.cc:374] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.529057 3426 raft_consensus.cc:729] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.529162 3426 consensus_queue.cc:260] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.529397 3426 ts_tablet_manager.cc:1428] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:38.529717 3476 ts_tablet_manager.cc:1918] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:38.529805 3476 log.cc:1196] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/eaacd35cc269468eb2964cb9be21f71b
I20260807 05:04:38.530124 3476 ts_tablet_manager.cc:1939] T eaacd35cc269468eb2964cb9be21f71b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting consensus metadata
I20260807 05:04:38.531983 3425 tablet_bootstrap.cc:492] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: No bootstrap required, opened a new log
I20260807 05:04:38.532058 3425 ts_tablet_manager.cc:1397] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260807 05:04:38.532456 3425 raft_consensus.cc:348] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.532555 3425 raft_consensus.cc:374] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:38.532582 3425 raft_consensus.cc:729] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9c1ef92ac14abaa5b394d9d0e7eb26, State: Initialized, Role: FOLLOWER
I20260807 05:04:38.532696 3425 consensus_queue.cc:260] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.532956 3425 ts_tablet_manager.cc:1428] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:38.533440 2891 catalog_manager.cc:4841] TS fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155): tablet eaacd35cc269468eb2964cb9be21f71b (table test-table [id=b71b4d2d704342ab957f1c2c8bf844cd]) successfully deleted
I20260807 05:04:38.535879 3475 ts_tablet_manager.cc:1905] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:38.539130 3475 ts_tablet_manager.cc:1918] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:38.539148 3474 ts_tablet_manager.cc:1918] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:38.539266 3475 log.cc:1196] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/eaacd35cc269468eb2964cb9be21f71b
I20260807 05:04:38.539376 3474 log.cc:1196] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/eaacd35cc269468eb2964cb9be21f71b
I20260807 05:04:38.539896 3474 ts_tablet_manager.cc:1939] T eaacd35cc269468eb2964cb9be21f71b P c4ffa30047604cff8d0d9601e523742d: Deleting consensus metadata
I20260807 05:04:38.540454 3475 ts_tablet_manager.cc:1939] T eaacd35cc269468eb2964cb9be21f71b P a8370a8803c44149814f04be9622e45d: Deleting consensus metadata
I20260807 05:04:38.541260 2889 catalog_manager.cc:4841] TS c4ffa30047604cff8d0d9601e523742d (127.1.17.193:44485): tablet eaacd35cc269468eb2964cb9be21f71b (table test-table [id=b71b4d2d704342ab957f1c2c8bf844cd]) successfully deleted
I20260807 05:04:38.541584 2891 catalog_manager.cc:4841] TS a8370a8803c44149814f04be9622e45d (127.1.17.194:41961): tablet eaacd35cc269468eb2964cb9be21f71b (table test-table [id=b71b4d2d704342ab957f1c2c8bf844cd]) successfully deleted
I20260807 05:04:38.635257 3470 raft_consensus.cc:482] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:38.635406 3470 raft_consensus.cc:504] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.635918 3470 leader_election.cc:288] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:38.636432 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899b552a65f4491a8c04db78afd9ed0b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" is_pre_election: true
I20260807 05:04:38.636489 3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899b552a65f4491a8c04db78afd9ed0b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d" is_pre_election: true
I20260807 05:04:38.636628 3354 raft_consensus.cc:2454] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:38.636709 3210 raft_consensus.cc:2454] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:38.637092 2999 leader_election.cc:302] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8370a8803c44149814f04be9622e45d, c4ffa30047604cff8d0d9601e523742d; no voters:
I20260807 05:04:38.637324 3470 raft_consensus.cc:2781] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:38.637404 3470 raft_consensus.cc:482] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:38.637458 3470 raft_consensus.cc:3037] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.638540 3470 raft_consensus.cc:504] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.638907 3470 leader_election.cc:288] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Requested vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:38.639331 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899b552a65f4491a8c04db78afd9ed0b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26"
I20260807 05:04:38.639338 3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899b552a65f4491a8c04db78afd9ed0b" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d"
I20260807 05:04:38.639469 3210 raft_consensus.cc:3037] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.639478 3354 raft_consensus.cc:3037] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:38.640614 3354 raft_consensus.cc:2454] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:38.640813 3210 raft_consensus.cc:2454] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:38.641059 2999 leader_election.cc:302] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4ffa30047604cff8d0d9601e523742d, fc9c1ef92ac14abaa5b394d9d0e7eb26; no voters:
I20260807 05:04:38.641297 3470 raft_consensus.cc:2781] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:38.641413 3470 raft_consensus.cc:686] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Becoming Leader. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Running, Role: LEADER
I20260807 05:04:38.641589 3470 consensus_queue.cc:237] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:38.643540 2904 catalog_manager.cc:5495] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d reported cstate change: term changed from 0 to 1, leader changed from <none> to c4ffa30047604cff8d0d9601e523742d (127.1.17.193). New cstate: current_term: 1 leader_uuid: "c4ffa30047604cff8d0d9601e523742d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } health_report { overall_health: UNKNOWN } } }
I20260807 05:04:38.687647 3354 raft_consensus.cc:1261] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:38.687700 3210 raft_consensus.cc:1261] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:38.688103 3436 consensus_queue.cc:1035] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:38.688262 3470 consensus_queue.cc:1035] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 05:04:38.702381 3456 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 05:04:40.253217 2903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:40.253365 2903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:40.255702 2903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Sending DeleteTablet for 3 replicas of tablet 899b552a65f4491a8c04db78afd9ed0b
I20260807 05:04:40.256337 3335 tablet_service.cc:1514] Processing DeleteTablet for tablet 899b552a65f4491a8c04db78afd9ed0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:40 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:47490
I20260807 05:04:40.256333 3045 tablet_service.cc:1514] Processing DeleteTablet for tablet 899b552a65f4491a8c04db78afd9ed0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:40 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60720
I20260807 05:04:40.256338 3191 tablet_service.cc:1514] Processing DeleteTablet for tablet 899b552a65f4491a8c04db78afd9ed0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:40 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:54360
I20260807 05:04:40.256678 3507 tablet_replica.cc:330] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: stopping tablet replica
I20260807 05:04:40.256867 3508 tablet_replica.cc:330] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: stopping tablet replica
I20260807 05:04:40.256865 3507 raft_consensus.cc:2229] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:40.257011 3507 raft_consensus.cc:2258] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:40.258448 3507 ts_tablet_manager.cc:1905] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:40.257076 3508 raft_consensus.cc:2229] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:40.260388 3508 raft_consensus.cc:2258] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:40.260679 3509 tablet_replica.cc:330] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: stopping tablet replica
I20260807 05:04:40.260860 3509 raft_consensus.cc:2229] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:40.260982 3507 ts_tablet_manager.cc:1918] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:04:40.261023 3509 raft_consensus.cc:2258] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:40.261049 3507 log.cc:1196] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/899b552a65f4491a8c04db78afd9ed0b
I20260807 05:04:40.261353 3507 ts_tablet_manager.cc:1939] T 899b552a65f4491a8c04db78afd9ed0b P a8370a8803c44149814f04be9622e45d: Deleting consensus metadata
I20260807 05:04:40.261358 3508 ts_tablet_manager.cc:1905] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:40.261677 3509 ts_tablet_manager.cc:1905] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:40.261966 2891 catalog_manager.cc:4841] TS a8370a8803c44149814f04be9622e45d (127.1.17.194:41961): tablet 899b552a65f4491a8c04db78afd9ed0b (table test-table [id=ddf36c48bfcd420988fe92d6ea005f99]) successfully deleted
I20260807 05:04:40.264189 3509 ts_tablet_manager.cc:1918] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:04:40.264189 3508 ts_tablet_manager.cc:1918] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:04:40.264272 3508 log.cc:1196] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/899b552a65f4491a8c04db78afd9ed0b
I20260807 05:04:40.264272 3509 log.cc:1196] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/899b552a65f4491a8c04db78afd9ed0b
I20260807 05:04:40.264569 3509 ts_tablet_manager.cc:1939] T 899b552a65f4491a8c04db78afd9ed0b P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting consensus metadata
I20260807 05:04:40.264583 3508 ts_tablet_manager.cc:1939] T 899b552a65f4491a8c04db78afd9ed0b P c4ffa30047604cff8d0d9601e523742d: Deleting consensus metadata
I20260807 05:04:40.265218 2889 catalog_manager.cc:4841] TS c4ffa30047604cff8d0d9601e523742d (127.1.17.193:44485): tablet 899b552a65f4491a8c04db78afd9ed0b (table test-table [id=ddf36c48bfcd420988fe92d6ea005f99]) successfully deleted
I20260807 05:04:40.265308 2891 catalog_manager.cc:4841] TS fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155): tablet 899b552a65f4491a8c04db78afd9ed0b (table test-table [id=ddf36c48bfcd420988fe92d6ea005f99]) successfully deleted
Aug 07 05:04:40 dist-test-slave-1x4g krb5kdc[2858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:04:40 dist-test-slave-1x4g krb5kdc[2858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:40.293897 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:40.279853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.1.17.254:43833)
0807 05:04:40.280165 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:40.280174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:40.280190 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:04:40.280339 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:40.280342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:40.280399 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:40.280488 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:40.281327 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:40.281912 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:40.282831 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:40.283083 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:40.283189 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:40.290986 (+ 7797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:40.290989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:40.290991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:40.291016 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:40.292300 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:40.292672 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:40.292675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:40.292676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:40.292749 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:40.293049 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:40.293051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:40.293052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:40.293233 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:40.293333 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:40.293542 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 05:04:40.293645 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":114,"threads_started":1}
I20260807 05:04:40.296348 2903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:04:40.296897 2903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:40.301256 3045 tablet_service.cc:1467] Processing CreateTablet for tablet 474f548deb25426ab7dbe0ea7ef09433 (DEFAULT_TABLE table=test-table [id=25a1a959a0e64594b6c92abd387937be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:40.301270 3191 tablet_service.cc:1467] Processing CreateTablet for tablet 474f548deb25426ab7dbe0ea7ef09433 (DEFAULT_TABLE table=test-table [id=25a1a959a0e64594b6c92abd387937be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:40.301559 3191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f548deb25426ab7dbe0ea7ef09433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:40.301559 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f548deb25426ab7dbe0ea7ef09433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:40.301523 3335 tablet_service.cc:1467] Processing CreateTablet for tablet 474f548deb25426ab7dbe0ea7ef09433 (DEFAULT_TABLE table=test-table [id=25a1a959a0e64594b6c92abd387937be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:40.301833 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f548deb25426ab7dbe0ea7ef09433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:40.303861 3518 tablet_bootstrap.cc:492] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Bootstrap starting.
I20260807 05:04:40.304163 3519 tablet_bootstrap.cc:492] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: Bootstrap starting.
I20260807 05:04:40.304411 3520 tablet_bootstrap.cc:492] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: Bootstrap starting.
I20260807 05:04:40.305020 3518 tablet_bootstrap.cc:654] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:40.305092 3519 tablet_bootstrap.cc:654] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:40.305347 3520 tablet_bootstrap.cc:654] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:40.306238 3518 tablet_bootstrap.cc:492] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: No bootstrap required, opened a new log
I20260807 05:04:40.306345 3518 ts_tablet_manager.cc:1397] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:40.306521 3520 tablet_bootstrap.cc:492] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: No bootstrap required, opened a new log
I20260807 05:04:40.306613 3520 ts_tablet_manager.cc:1397] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 05:04:40.306775 3518 raft_consensus.cc:348] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.306902 3518 raft_consensus.cc:374] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:40.306951 3518 raft_consensus.cc:729] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9c1ef92ac14abaa5b394d9d0e7eb26, State: Initialized, Role: FOLLOWER
I20260807 05:04:40.307089 3518 consensus_queue.cc:260] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.307072 3520 raft_consensus.cc:348] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.307194 3520 raft_consensus.cc:374] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:40.307245 3520 raft_consensus.cc:729] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8370a8803c44149814f04be9622e45d, State: Initialized, Role: FOLLOWER
I20260807 05:04:40.307380 3518 ts_tablet_manager.cc:1428] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:40.307389 3520 consensus_queue.cc:260] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.307839 3520 ts_tablet_manager.cc:1428] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:04:40.308096 3519 tablet_bootstrap.cc:492] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: No bootstrap required, opened a new log
I20260807 05:04:40.308219 3519 ts_tablet_manager.cc:1397] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 05:04:40.308699 3519 raft_consensus.cc:348] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.308841 3519 raft_consensus.cc:374] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:40.308889 3519 raft_consensus.cc:729] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Initialized, Role: FOLLOWER
I20260807 05:04:40.309057 3519 consensus_queue.cc:260] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.309655 3519 ts_tablet_manager.cc:1428] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:04:40.437940 3485 raft_consensus.cc:482] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:40.438107 3485 raft_consensus.cc:504] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.438530 3485 leader_election.cc:288] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:40.439054 3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f548deb25426ab7dbe0ea7ef09433" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d" is_pre_election: true
I20260807 05:04:40.439083 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f548deb25426ab7dbe0ea7ef09433" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" is_pre_election: true
I20260807 05:04:40.439234 3210 raft_consensus.cc:2454] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:40.439244 3354 raft_consensus.cc:2454] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:40.439603 2999 leader_election.cc:302] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8370a8803c44149814f04be9622e45d, c4ffa30047604cff8d0d9601e523742d; no voters:
I20260807 05:04:40.439821 3485 raft_consensus.cc:2781] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:40.439891 3485 raft_consensus.cc:482] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:40.439942 3485 raft_consensus.cc:3037] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:40.441125 3485 raft_consensus.cc:504] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.441429 3485 leader_election.cc:288] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Requested vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:40.441794 3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f548deb25426ab7dbe0ea7ef09433" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d"
I20260807 05:04:40.441928 3210 raft_consensus.cc:3037] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:40.441900 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f548deb25426ab7dbe0ea7ef09433" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26"
I20260807 05:04:40.442126 3354 raft_consensus.cc:3037] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:40.442845 3210 raft_consensus.cc:2454] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:40.443116 3354 raft_consensus.cc:2454] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:40.443145 2999 leader_election.cc:302] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8370a8803c44149814f04be9622e45d, c4ffa30047604cff8d0d9601e523742d; no voters:
I20260807 05:04:40.443364 3485 raft_consensus.cc:2781] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:40.443507 3485 raft_consensus.cc:686] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Becoming Leader. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Running, Role: LEADER
I20260807 05:04:40.443652 3485 consensus_queue.cc:237] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.445335 2902 catalog_manager.cc:5495] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d reported cstate change: term changed from 0 to 1, leader changed from <none> to c4ffa30047604cff8d0d9601e523742d (127.1.17.193). New cstate: current_term: 1 leader_uuid: "c4ffa30047604cff8d0d9601e523742d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } health_report { overall_health: UNKNOWN } } }
I20260807 05:04:40.465808 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:40.462504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41524 (local address 127.1.17.193:44485)
0807 05:04:40.462778 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:40.462787 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:40.462799 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:04:40.462843 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:40.462845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:40.462881 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:40.462996 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:40.463249 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:40.463787 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:40.464661 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:40.464975 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:40.465082 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:40.465357 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:40.465440 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:40.465521 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:40.465608 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 05:04:40.465663 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":100,"threads_started":1}
I20260807 05:04:40.471796 3210 raft_consensus.cc:1261] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:40.471786 3354 raft_consensus.cc:1261] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:40.472185 3485 consensus_queue.cc:1035] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:40.472357 3502 consensus_queue.cc:1035] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:40.502529 2904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:40.502673 2904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:40.504460 2904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Sending DeleteTablet for 3 replicas of tablet 474f548deb25426ab7dbe0ea7ef09433
I20260807 05:04:40.504972 3045 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f548deb25426ab7dbe0ea7ef09433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:40 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60720
I20260807 05:04:40.505199 3191 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f548deb25426ab7dbe0ea7ef09433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:40 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:54360
I20260807 05:04:40.505300 3335 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f548deb25426ab7dbe0ea7ef09433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:40 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:47490
I20260807 05:04:40.505426 3508 tablet_replica.cc:330] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: stopping tablet replica
I20260807 05:04:40.505498 3509 tablet_replica.cc:330] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: stopping tablet replica
I20260807 05:04:40.505558 3508 raft_consensus.cc:2229] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:40.505635 3509 raft_consensus.cc:2229] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:40.505743 3509 raft_consensus.cc:2258] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:40.505764 3508 raft_consensus.cc:2258] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:40.505939 3507 tablet_replica.cc:330] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: stopping tablet replica
I20260807 05:04:40.506070 3507 raft_consensus.cc:2229] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:40.506029 2904 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:04:40.506203 3507 raft_consensus.cc:2258] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 05:04:40.506444 2904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:40.506640 3509 ts_tablet_manager.cc:1905] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:40.506840 3507 ts_tablet_manager.cc:1905] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:40.506862 3508 ts_tablet_manager.cc:1905] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:40.509016 3509 ts_tablet_manager.cc:1918] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:40.509073 3509 log.cc:1196] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/474f548deb25426ab7dbe0ea7ef09433
I20260807 05:04:40.509263 3509 ts_tablet_manager.cc:1939] T 474f548deb25426ab7dbe0ea7ef09433 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting consensus metadata
I20260807 05:04:40.509384 3508 ts_tablet_manager.cc:1918] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:40.509384 3507 ts_tablet_manager.cc:1918] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:04:40.509454 3508 log.cc:1196] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/474f548deb25426ab7dbe0ea7ef09433
I20260807 05:04:40.509467 3507 log.cc:1196] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/474f548deb25426ab7dbe0ea7ef09433
I20260807 05:04:40.509716 3507 ts_tablet_manager.cc:1939] T 474f548deb25426ab7dbe0ea7ef09433 P a8370a8803c44149814f04be9622e45d: Deleting consensus metadata
I20260807 05:04:40.509716 3508 ts_tablet_manager.cc:1939] T 474f548deb25426ab7dbe0ea7ef09433 P c4ffa30047604cff8d0d9601e523742d: Deleting consensus metadata
I20260807 05:04:40.510049 2891 catalog_manager.cc:4841] TS fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155): tablet 474f548deb25426ab7dbe0ea7ef09433 (table test-table [id=25a1a959a0e64594b6c92abd387937be]) successfully deleted
I20260807 05:04:40.510440 2889 catalog_manager.cc:4841] TS c4ffa30047604cff8d0d9601e523742d (127.1.17.193:44485): tablet 474f548deb25426ab7dbe0ea7ef09433 (table test-table [id=25a1a959a0e64594b6c92abd387937be]) successfully deleted
I20260807 05:04:40.510581 2891 catalog_manager.cc:4841] TS a8370a8803c44149814f04be9622e45d (127.1.17.194:41961): tablet 474f548deb25426ab7dbe0ea7ef09433 (table test-table [id=25a1a959a0e64594b6c92abd387937be]) successfully deleted
I20260807 05:04:40.513818 3045 tablet_service.cc:1467] Processing CreateTablet for tablet 69fd3cac8e144f90a66504c66db8eed8 (DEFAULT_TABLE table=test-table [id=af73a7831ea24e049d97680e6f5582c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:40.513955 3335 tablet_service.cc:1467] Processing CreateTablet for tablet 69fd3cac8e144f90a66504c66db8eed8 (DEFAULT_TABLE table=test-table [id=af73a7831ea24e049d97680e6f5582c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:40.514099 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69fd3cac8e144f90a66504c66db8eed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:40.514091 3191 tablet_service.cc:1467] Processing CreateTablet for tablet 69fd3cac8e144f90a66504c66db8eed8 (DEFAULT_TABLE table=test-table [id=af73a7831ea24e049d97680e6f5582c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:40.514204 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69fd3cac8e144f90a66504c66db8eed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:40.514358 3191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69fd3cac8e144f90a66504c66db8eed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:40.516261 3520 tablet_bootstrap.cc:492] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: Bootstrap starting.
I20260807 05:04:40.516489 3518 tablet_bootstrap.cc:492] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Bootstrap starting.
I20260807 05:04:40.517315 3520 tablet_bootstrap.cc:654] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:40.517377 3518 tablet_bootstrap.cc:654] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:40.518221 3519 tablet_bootstrap.cc:492] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d: Bootstrap starting.
I20260807 05:04:40.518533 3518 tablet_bootstrap.cc:492] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: No bootstrap required, opened a new log
I20260807 05:04:40.518628 3518 ts_tablet_manager.cc:1397] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:40.518953 3520 tablet_bootstrap.cc:492] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: No bootstrap required, opened a new log
I20260807 05:04:40.519052 3520 ts_tablet_manager.cc:1397] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:40.519093 3519 tablet_bootstrap.cc:654] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:40.519030 3518 raft_consensus.cc:348] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.519147 3518 raft_consensus.cc:374] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:40.519181 3518 raft_consensus.cc:729] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9c1ef92ac14abaa5b394d9d0e7eb26, State: Initialized, Role: FOLLOWER
I20260807 05:04:40.519310 3518 consensus_queue.cc:260] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.519537 3520 raft_consensus.cc:348] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.519609 3518 ts_tablet_manager.cc:1428] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:40.519659 3520 raft_consensus.cc:374] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:40.519701 3520 raft_consensus.cc:729] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8370a8803c44149814f04be9622e45d, State: Initialized, Role: FOLLOWER
I20260807 05:04:40.519886 3520 consensus_queue.cc:260] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.520175 3520 ts_tablet_manager.cc:1428] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:04:40.520314 3519 tablet_bootstrap.cc:492] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d: No bootstrap required, opened a new log
I20260807 05:04:40.520395 3519 ts_tablet_manager.cc:1397] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 05:04:40.520828 3519 raft_consensus.cc:348] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.520933 3519 raft_consensus.cc:374] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:40.520956 3519 raft_consensus.cc:729] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Initialized, Role: FOLLOWER
I20260807 05:04:40.521075 3519 consensus_queue.cc:260] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.521337 3519 ts_tablet_manager.cc:1428] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:04:40.532349 3485 raft_consensus.cc:482] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:40.532433 3485 raft_consensus.cc:504] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.532863 3485 leader_election.cc:288] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:40.533212 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fd3cac8e144f90a66504c66db8eed8" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d" is_pre_election: true
I20260807 05:04:40.533274 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fd3cac8e144f90a66504c66db8eed8" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" is_pre_election: true
I20260807 05:04:40.533368 3211 raft_consensus.cc:2454] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:40.533378 3354 raft_consensus.cc:2454] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 0.
I20260807 05:04:40.533615 2999 leader_election.cc:302] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8370a8803c44149814f04be9622e45d, c4ffa30047604cff8d0d9601e523742d; no voters:
I20260807 05:04:40.533775 3485 raft_consensus.cc:2781] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:40.533835 3485 raft_consensus.cc:482] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:40.533879 3485 raft_consensus.cc:3037] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:40.534956 3485 raft_consensus.cc:504] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.535244 3485 leader_election.cc:288] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Requested vote from peers a8370a8803c44149814f04be9622e45d (127.1.17.194:41961), fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155)
I20260807 05:04:40.535540 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fd3cac8e144f90a66504c66db8eed8" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8370a8803c44149814f04be9622e45d"
I20260807 05:04:40.535605 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69fd3cac8e144f90a66504c66db8eed8" candidate_uuid: "c4ffa30047604cff8d0d9601e523742d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26"
I20260807 05:04:40.535696 3211 raft_consensus.cc:3037] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:40.535709 3354 raft_consensus.cc:3037] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:40.536522 3354 raft_consensus.cc:2454] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:40.536595 3211 raft_consensus.cc:2454] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ffa30047604cff8d0d9601e523742d in term 1.
I20260807 05:04:40.536857 2999 leader_election.cc:302] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4ffa30047604cff8d0d9601e523742d, fc9c1ef92ac14abaa5b394d9d0e7eb26; no voters:
I20260807 05:04:40.537060 3485 raft_consensus.cc:2781] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:40.537171 3485 raft_consensus.cc:686] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Becoming Leader. State: Replica: c4ffa30047604cff8d0d9601e523742d, State: Running, Role: LEADER
I20260807 05:04:40.537300 3485 consensus_queue.cc:237] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } }
I20260807 05:04:40.538641 2905 catalog_manager.cc:5495] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d reported cstate change: term changed from 0 to 1, leader changed from <none> to c4ffa30047604cff8d0d9601e523742d (127.1.17.193). New cstate: current_term: 1 leader_uuid: "c4ffa30047604cff8d0d9601e523742d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4ffa30047604cff8d0d9601e523742d" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 } health_report { overall_health: UNKNOWN } } }
I20260807 05:04:40.559957 3354 raft_consensus.cc:1261] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:40.559985 3211 raft_consensus.cc:1261] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Refusing update from remote peer c4ffa30047604cff8d0d9601e523742d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:40.560323 3498 consensus_queue.cc:1035] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc9c1ef92ac14abaa5b394d9d0e7eb26" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:40.560495 3485 consensus_queue.cc:1035] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8370a8803c44149814f04be9622e45d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 05:04:40.573724 3490 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 05:04:42.122089 2905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:42.122236 2905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:42.124488 2905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21eafe375d514aec8084e156b5ca1749: Sending DeleteTablet for 3 replicas of tablet 69fd3cac8e144f90a66504c66db8eed8
I20260807 05:04:42.125029 3045 tablet_service.cc:1514] Processing DeleteTablet for tablet 69fd3cac8e144f90a66504c66db8eed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:42 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60720
I20260807 05:04:42.125097 3191 tablet_service.cc:1514] Processing DeleteTablet for tablet 69fd3cac8e144f90a66504c66db8eed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:42 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:54360
I20260807 05:04:42.125380 3562 tablet_replica.cc:330] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d: stopping tablet replica
I20260807 05:04:42.125470 3563 tablet_replica.cc:330] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: stopping tablet replica
I20260807 05:04:42.125555 3562 raft_consensus.cc:2229] T 69fd3cac8e144f90a66504c66db8eed8 P c4ffa30047604cff8d0d9601e523742d [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:42.125638 3563 raft_consensus.cc:2229] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:42.125739 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 2979
I20260807 05:04:42.125787 3563 raft_consensus.cc:2258] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:42.126277 3335 tablet_service.cc:1514] Processing DeleteTablet for tablet 69fd3cac8e144f90a66504c66db8eed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:42 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:47490
I20260807 05:04:42.127441 3564 tablet_replica.cc:330] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: stopping tablet replica
I20260807 05:04:42.127625 3564 raft_consensus.cc:2229] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:42.127776 3564 raft_consensus.cc:2258] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:42.128355 3564 ts_tablet_manager.cc:1905] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:42.129096 3563 ts_tablet_manager.cc:1905] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:42.130718 3564 ts_tablet_manager.cc:1918] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:04:42.130811 3564 log.cc:1196] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/69fd3cac8e144f90a66504c66db8eed8
I20260807 05:04:42.131091 3564 ts_tablet_manager.cc:1939] T 69fd3cac8e144f90a66504c66db8eed8 P fc9c1ef92ac14abaa5b394d9d0e7eb26: Deleting consensus metadata
I20260807 05:04:42.131767 2891 catalog_manager.cc:4841] TS fc9c1ef92ac14abaa5b394d9d0e7eb26 (127.1.17.195:44155): tablet 69fd3cac8e144f90a66504c66db8eed8 (table test-table [id=af73a7831ea24e049d97680e6f5582c4]) successfully deleted
I20260807 05:04:42.132397 3563 ts_tablet_manager.cc:1918] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 05:04:42.132491 3563 log.cc:1196] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/69fd3cac8e144f90a66504c66db8eed8
I20260807 05:04:42.132756 3563 ts_tablet_manager.cc:1939] T 69fd3cac8e144f90a66504c66db8eed8 P a8370a8803c44149814f04be9622e45d: Deleting consensus metadata
I20260807 05:04:42.133541 2891 catalog_manager.cc:4841] TS a8370a8803c44149814f04be9622e45d (127.1.17.194:41961): tablet 69fd3cac8e144f90a66504c66db8eed8 (table test-table [id=af73a7831ea24e049d97680e6f5582c4]) successfully deleted
I20260807 05:04:42.137379 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 3123
W20260807 05:04:42.137475 2889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:44485 (error 108)
I20260807 05:04:42.139900 3566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:42.139394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44485 (local address 127.0.0.1:41532)
0807 05:04:42.139680 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:42.139779 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:44485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":70,"threads_started":1}
W20260807 05:04:42.140053 2889 catalog_manager.cc:4584] TS c4ffa30047604cff8d0d9601e523742d (127.1.17.193:44485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69fd3cac8e144f90a66504c66db8eed8: Network error: Client connection negotiation failed: client connection to 127.1.17.193:44485: connect: Connection refused (error 111)
I20260807 05:04:42.144553 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 3268
I20260807 05:04:42.152132 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 2872
2026-08-07T05:04:42Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6220 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:42 dist-test-slave-1x4g krb5kdc[3569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:42 dist-test-slave-1x4g krb5kdc[3569](info): set up 2 sockets
Aug 07 05:04:42 dist-test-slave-1x4g krb5kdc[3569](info): commencing operation
krb5kdc: starting...
W20260807 05:04:43.190397 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:43 dist-test-slave-1x4g krb5kdc[3569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:43.315904 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:42551
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:42551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:43.425781 3583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:43.426048 3583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:43.426088 3583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:43.429970 3583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:43.430020 3583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:43.430038 3583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:43.430054 3583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:43.430094 3583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:43.435205 3583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:42551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:42551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:43.436199 3583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:43.437284 3583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:43.445477 3588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:43.445526 3589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:43.445748 3583 server_base.cc:1034] running on GCE node
W20260807 05:04:43.445660 3591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:43.446455 3583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:43.447504 3583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:43.448691 3583 hybrid_clock.cc:648] HybridClock initialized: now 1786079083448672 us; error 29 us; skew 500 ppm
Aug 07 05:04:43 dist-test-slave-1x4g krb5kdc[3569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079083, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:43.451499 3583 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:43.452332 3583 webserver.cc:460] Webserver started at http://127.1.17.254:36517/ using document root <none> and password file <none>
I20260807 05:04:43.452945 3583 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:43.453039 3583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:43.453274 3583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:43.455267 3583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "35bb60c9c50b4bd1a19745b4dc42783b"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "5609857a0e203260e3041ec5a1d89e40"
server_key_iv: "17f0602fdef65e7b83ea413342479c51"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.455854 3583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "35bb60c9c50b4bd1a19745b4dc42783b"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "5609857a0e203260e3041ec5a1d89e40"
server_key_iv: "17f0602fdef65e7b83ea413342479c51"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.459580 3583 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:04:43.462356 3598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:43.463619 3583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:43.463716 3583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "35bb60c9c50b4bd1a19745b4dc42783b"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "5609857a0e203260e3041ec5a1d89e40"
server_key_iv: "17f0602fdef65e7b83ea413342479c51"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.463794 3583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:43.493717 3583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:43.496563 3583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:43.496745 3583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:43.503762 3583 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:42551
I20260807 05:04:43.503774 3650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:42551 every 8 connection(s)
I20260807 05:04:43.504716 3583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:43.507524 3651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:43.511958 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 3583
I20260807 05:04:43.512065 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:43.512315 1095 external_mini_cluster.cc:1428] Setting key 7c23af50240a184ac92e34ef8bf2b46a
I20260807 05:04:43.513293 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b: Bootstrap starting.
I20260807 05:04:43.515630 3651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:43.516551 3651 log.cc:824] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:43.518643 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b: No bootstrap required, opened a new log
Aug 07 05:04:43 dist-test-slave-1x4g krb5kdc[3569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:43.523043 3651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42551 } }
I20260807 05:04:43.523294 3651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:43.523380 3651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bb60c9c50b4bd1a19745b4dc42783b, State: Initialized, Role: FOLLOWER
I20260807 05:04:43.523809 3651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42551 } }
I20260807 05:04:43.523979 3651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:43.524065 3651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:43.524184 3651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:43.525370 3651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42551 } }
I20260807 05:04:43.525702 3651 leader_election.cc:302] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35bb60c9c50b4bd1a19745b4dc42783b; no voters:
I20260807 05:04:43.526074 3651 leader_election.cc:288] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:43.526214 3656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:43.526496 3656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [term 1 LEADER]: Becoming Leader. State: Replica: 35bb60c9c50b4bd1a19745b4dc42783b, State: Running, Role: LEADER
I20260807 05:04:43.526809 3656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42551 } }
I20260807 05:04:43.527341 3651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:43.527537 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:43.513701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33192 (local address 127.1.17.254:42551)
0807 05:04:43.514180 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:43.514196 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:43.514231 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:04:43.514843 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:43.514859 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:43.515188 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:43.515465 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:43.516428 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:43.517701 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:43.518831 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:43.519278 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:43.519391 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:43.522109 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:43.522114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:43.522127 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:43.522152 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:43.524231 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:43.524731 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:43.524736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:43.524740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:43.524935 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:43.525253 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:43.525256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:43.525257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:43.525692 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:43.525890 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:43.526258 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 05:04:43.526578 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":167,"threads_started":1}
I20260807 05:04:43.528009 3656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42551 } } }
I20260807 05:04:43.528199 3656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:43.528403 3656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35bb60c9c50b4bd1a19745b4dc42783b. Latest consensus state: current_term: 1 leader_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bb60c9c50b4bd1a19745b4dc42783b" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42551 } } }
I20260807 05:04:43.528527 3656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:43.529780 3663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:43.532553 3663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:43.538008 3663 catalog_manager.cc:1347] Generated new cluster ID: 427426fa16f042adaa3c4ac6bcaf8ff1
I20260807 05:04:43.538071 3663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:43.554589 3663 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:43.555917 3663 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:43.563688 3663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35bb60c9c50b4bd1a19745b4dc42783b: Generated new TSK 0
I20260807 05:04:43.564363 3663 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:43.597777 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42551
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:43.709744 3677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:43.709985 3677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:43.710026 3677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:43.713961 3677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:43.714010 3677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:43.714063 3677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:43.719019 3677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:43.719889 3677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:43.720834 3677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:43.730628 3683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:43.730820 3677 server_base.cc:1034] running on GCE node
W20260807 05:04:43.730660 3685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:43.730674 3682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:43.731395 3677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:43.731976 3677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:43.733137 3677 hybrid_clock.cc:648] HybridClock initialized: now 1786079083733115 us; error 31 us; skew 500 ppm
Aug 07 05:04:43 dist-test-slave-1x4g krb5kdc[3569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079083, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:43.735816 3677 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:43.736726 3677 webserver.cc:460] Webserver started at http://127.1.17.193:43013/ using document root <none> and password file <none>
I20260807 05:04:43.737293 3677 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:43.737385 3677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:43.737612 3677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:43.739610 3677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "99c869398570402ba3eda86a4af1758b"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "3746d3185721eb51efb45453c947030b"
server_key_iv: "cd7948530d866fd826058845629fe9e6"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.740180 3677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "99c869398570402ba3eda86a4af1758b"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "3746d3185721eb51efb45453c947030b"
server_key_iv: "cd7948530d866fd826058845629fe9e6"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.743721 3677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:04:43.746217 3692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:43.747419 3677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:43.747584 3677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "99c869398570402ba3eda86a4af1758b"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "3746d3185721eb51efb45453c947030b"
server_key_iv: "cd7948530d866fd826058845629fe9e6"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.747695 3677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:43.768580 3677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:43.771883 3677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:43.772090 3677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:43.772684 3677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:43.773524 3677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:43.773594 3677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:43.773681 3677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:43.773756 3677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:43.782456 3677 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:39523
I20260807 05:04:43.782487 3808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:39523 every 8 connection(s)
I20260807 05:04:43.783344 3677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:43.784086 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 3677
I20260807 05:04:43.784195 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:43.784547 1095 external_mini_cluster.cc:1428] Setting key 1d6cf9327d0bc17bc59e7e79e36d2921
Aug 07 05:04:43 dist-test-slave-1x4g krb5kdc[3569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079083, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:04:43.799371 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:43.785363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:43551 (local address 127.1.17.254:42551)
0807 05:04:43.785512 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:43.785515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:43.786598 (+ 1083us) server_negotiation.cc:407] Connection header received
0807 05:04:43.788716 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:43.788722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:43.788831 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:43.788966 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:43.790147 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:43.790712 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:43.792569 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:43.792864 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:43.792955 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:43.795805 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:43.795807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:43.795810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:43.795833 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:43.797292 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:43.797816 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:43.797820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:43.797822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:43.797904 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:43.798254 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:43.798258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:43.798260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:43.798457 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:43.798555 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:43.799068 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 05:04:43.799194 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:04:43.800237 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:43.785863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:42551 (local address 127.1.17.193:43551)
0807 05:04:43.786447 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:43.786506 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:43.788445 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:43.789148 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:43.789155 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:43.789493 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:43.789984 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:43.789998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:43.790881 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:43.790888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:43.792442 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:43.792458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:43.793044 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:43.793052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:43.793284 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:43.793934 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:43.793958 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:43.795633 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:43.797424 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:43.797429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:43.797443 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:43.797709 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:43.798009 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:43.798012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:43.798013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:43.798157 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:43.798592 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:43.798599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:43.798828 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 05:04:43.799019 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 05:04:43.799332 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":175,"threads_started":1}
I20260807 05:04:43.801512 3809 heartbeater.cc:347] Connected to a master server at 127.1.17.254:42551
I20260807 05:04:43.801791 3809 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:43.802474 3809 heartbeater.cc:511] Master 127.1.17.254:42551 requested a full tablet report, sending...
I20260807 05:04:43.804234 3615 ts_manager.cc:194] Registered new tserver with Master: 99c869398570402ba3eda86a4af1758b (127.1.17.193:39523)
I20260807 05:04:43.805976 3615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:43551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:43.817773 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42551
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:43.930053 3814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:43.930275 3814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:43.930316 3814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:43.934293 3814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:43.934345 3814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:43.934401 3814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:43.939476 3814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:43.940368 3814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:43.941291 3814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:43.950727 3822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:43.950724 3819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:43.950749 3820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:43.951042 3814 server_base.cc:1034] running on GCE node
I20260807 05:04:43.951536 3814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:43.952090 3814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:43.953280 3814 hybrid_clock.cc:648] HybridClock initialized: now 1786079083953240 us; error 51 us; skew 500 ppm
Aug 07 05:04:43 dist-test-slave-1x4g krb5kdc[3569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079083, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:43.956173 3814 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:43.957087 3814 webserver.cc:460] Webserver started at http://127.1.17.194:44401/ using document root <none> and password file <none>
I20260807 05:04:43.957633 3814 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:43.957696 3814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:43.957864 3814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:43.959863 3814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "09936addc03249f3b921a25f04242812"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "dc66929e704c670321fd54b5e7615d0c"
server_key_iv: "35d59c1fb8444009d9dc04b39b62041e"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.960379 3814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "09936addc03249f3b921a25f04242812"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "dc66929e704c670321fd54b5e7615d0c"
server_key_iv: "35d59c1fb8444009d9dc04b39b62041e"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.964685 3814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:04:43.967343 3829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:43.968850 3814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:04:43.968968 3814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "09936addc03249f3b921a25f04242812"
format_stamp: "Formatted at 2026-08-07 05:04:43 on dist-test-slave-1x4g"
server_key: "dc66929e704c670321fd54b5e7615d0c"
server_key_iv: "35d59c1fb8444009d9dc04b39b62041e"
server_key_version: "encryptionkey@0"
I20260807 05:04:43.969054 3814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:43.983413 3814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:43.986267 3814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:43.986423 3814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:43.986931 3814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:43.987701 3814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:43.987747 3814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:43.987797 3814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:43.987823 3814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:43.997455 3814 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:34103
I20260807 05:04:43.997490 3944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:34103 every 8 connection(s)
I20260807 05:04:43.998309 3814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:44.004382 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 3814
I20260807 05:04:44.004516 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:44.004851 1095 external_mini_cluster.cc:1428] Setting key f64cb8b45a664d290bd77e9fcd4b7726
Aug 07 05:04:44 dist-test-slave-1x4g krb5kdc[3569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079083, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:44.013195 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:44.000092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:48607 (local address 127.1.17.254:42551)
0807 05:04:44.000227 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:44.000231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:44.001087 (+ 856us) server_negotiation.cc:407] Connection header received
0807 05:04:44.002344 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:44.002347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:44.002400 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:44.002474 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:44.003617 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.004168 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.005789 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.006131 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.006243 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:44.009353 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:44.009356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:44.009358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:44.009382 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:44.010875 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:44.011443 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:44.011446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:44.011449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:44.011533 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:44.012005 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:44.012007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:44.012008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:44.012183 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:44.012259 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:44.012862 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 05:04:44.013028 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:04:44.013962 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:44.000465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:42551 (local address 127.1.17.194:48607)
0807 05:04:44.000948 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:44.000990 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:44.002145 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:44.002638 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:44.002644 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:44.002991 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:44.003477 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:44.003491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.004311 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.004315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:44.005668 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:44.005679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.006303 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.006306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:44.006527 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:44.007103 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:44.007128 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:44.009177 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:44.011054 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:44.011060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:44.011080 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:44.011333 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:44.011672 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:44.011679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:44.011683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:44.011892 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:44.012292 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:44.012298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:44.012577 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 05:04:44.012769 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 05:04:44.013114 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260807 05:04:44.015125 3945 heartbeater.cc:347] Connected to a master server at 127.1.17.254:42551
I20260807 05:04:44.015372 3945 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:04:44.015843 3945 heartbeater.cc:511] Master 127.1.17.254:42551 requested a full tablet report, sending...
I20260807 05:04:44.016960 3615 ts_manager.cc:194] Registered new tserver with Master: 09936addc03249f3b921a25f04242812 (127.1.17.194:34103)
I20260807 05:04:44.017547 3615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:48607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:44.035533 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42551
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:44.149590 3950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:44.149820 3950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:44.149863 3950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:44.153900 3950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:44.153972 3950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:44.154028 3950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:44.159058 3950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:44.160032 3950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:44.161023 3950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:44.170872 3958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:44.170852 3956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:44.171041 3955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:44.171290 3950 server_base.cc:1034] running on GCE node
I20260807 05:04:44.171718 3950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:44.172400 3950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:44.173595 3950 hybrid_clock.cc:648] HybridClock initialized: now 1786079084173570 us; error 55 us; skew 500 ppm
Aug 07 05:04:44 dist-test-slave-1x4g krb5kdc[3569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079084, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:44.176729 3950 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:44.177870 3950 webserver.cc:460] Webserver started at http://127.1.17.195:41827/ using document root <none> and password file <none>
I20260807 05:04:44.178480 3950 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:44.178576 3950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:44.178817 3950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:44.181129 3950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "7c4440cf136747f9afb8f61b44522d00"
format_stamp: "Formatted at 2026-08-07 05:04:44 on dist-test-slave-1x4g"
server_key: "169f479f751e36d763cf6c7a07cb8ee9"
server_key_iv: "ef8e1751bcdab145b47b41b79f4fa019"
server_key_version: "encryptionkey@0"
I20260807 05:04:44.181823 3950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "7c4440cf136747f9afb8f61b44522d00"
format_stamp: "Formatted at 2026-08-07 05:04:44 on dist-test-slave-1x4g"
server_key: "169f479f751e36d763cf6c7a07cb8ee9"
server_key_iv: "ef8e1751bcdab145b47b41b79f4fa019"
server_key_version: "encryptionkey@0"
I20260807 05:04:44.186040 3950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 05:04:44.188716 3965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:44.190055 3950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:04:44.190232 3950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "7c4440cf136747f9afb8f61b44522d00"
format_stamp: "Formatted at 2026-08-07 05:04:44 on dist-test-slave-1x4g"
server_key: "169f479f751e36d763cf6c7a07cb8ee9"
server_key_iv: "ef8e1751bcdab145b47b41b79f4fa019"
server_key_version: "encryptionkey@0"
I20260807 05:04:44.190359 3950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:44.224579 3950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:44.227593 3950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:44.227820 3950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:44.228487 3950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:44.229404 3950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:44.229452 3950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:44.229492 3950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:44.229507 3950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:44.238809 3950 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:44499
I20260807 05:04:44.238849 4080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:44499 every 8 connection(s)
I20260807 05:04:44.239749 3950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:44.242601 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 3950
I20260807 05:04:44.242762 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:44.243088 1095 external_mini_cluster.cc:1428] Setting key 3cb56db55f341cfd49e546502de1a4c3
Aug 07 05:04:44 dist-test-slave-1x4g krb5kdc[3569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079084, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:44.253749 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:44.241615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:38953 (local address 127.1.17.254:42551)
0807 05:04:44.241784 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:44.241789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:44.242742 (+ 953us) server_negotiation.cc:407] Connection header received
0807 05:04:44.244519 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:44.244522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:44.244576 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:44.244674 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:44.246105 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.246649 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.247784 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.247998 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.248073 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:44.250558 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:44.250561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:44.250563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:44.250585 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:44.251937 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:44.252467 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:44.252469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:44.252471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:44.252546 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:44.252856 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:44.252858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:44.252860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:44.253038 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:44.253111 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:44.253543 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 05:04:44.253635 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 05:04:44.254550 4083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:44.242044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:42551 (local address 127.1.17.195:38953)
0807 05:04:44.242552 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:44.242608 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:44.244280 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:44.244893 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:44.244902 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:44.245469 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:44.245975 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:44.245989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.246766 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.246769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:44.247679 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:44.247688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:44.248120 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:44.248123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:44.248266 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:44.248907 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:44.248929 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:44.250402 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:44.252041 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:44.252046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:44.252057 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:44.252377 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:44.252623 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:44.252625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:44.252627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:44.252772 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:44.253122 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:44.253127 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:44.253331 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 05:04:44.253486 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 05:04:44.253715 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
I20260807 05:04:44.255748 4081 heartbeater.cc:347] Connected to a master server at 127.1.17.254:42551
I20260807 05:04:44.255995 4081 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:44.256625 4081 heartbeater.cc:511] Master 127.1.17.254:42551 requested a full tablet report, sending...
I20260807 05:04:44.257690 3615 ts_manager.cc:194] Registered new tserver with Master: 7c4440cf136747f9afb8f61b44522d00 (127.1.17.195:44499)
I20260807 05:04:44.258222 3615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:38953
I20260807 05:04:44.269210 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 05:04:44.285152 3654 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:44.283634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.1.17.254:42551)
0807 05:04:44.283800 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:44.283804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:44.283933 (+ 129us) server_negotiation.cc:407] Connection header received
0807 05:04:44.284056 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:44.284059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:44.284104 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:44.284182 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:44.284998 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33204: BlockingRecv error: recv got EOF from 127.0.0.1:33204 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 05:04:44.286252 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 3677
I20260807 05:04:44.293982 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 3814
I20260807 05:04:44.300827 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 3950
I20260807 05:04:44.307819 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 3583
2026-08-07T05:04:44Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2154 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:44 dist-test-slave-1x4g krb5kdc[4093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:44 dist-test-slave-1x4g krb5kdc[4093](info): set up 2 sockets
Aug 07 05:04:44 dist-test-slave-1x4g krb5kdc[4093](info): commencing operation
krb5kdc: starting...
W20260807 05:04:45.358553 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:45 dist-test-slave-1x4g krb5kdc[4093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:45.481896 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36257
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:36257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:45.591346 4106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:45.591578 4106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:45.591645 4106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:45.595606 4106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:45.595654 4106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:45.595671 4106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:45.595727 4106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:45.595758 4106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:45.600907 4106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:36257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:45.601859 4106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:45.602851 4106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:45.612247 4111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:45.613353 4114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:45.613507 4112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:45.613721 4106 server_base.cc:1034] running on GCE node
I20260807 05:04:45.614333 4106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:45.615595 4106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:45.618918 4106 hybrid_clock.cc:648] HybridClock initialized: now 1786079085618897 us; error 38 us; skew 500 ppm
Aug 07 05:04:45 dist-test-slave-1x4g krb5kdc[4093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079085, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:45.625895 4106 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:45.626784 4106 webserver.cc:460] Webserver started at http://127.1.17.254:43781/ using document root <none> and password file <none>
I20260807 05:04:45.627345 4106 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:45.627442 4106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:45.627661 4106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:45.629680 4106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "5824c2216b7f4f8bbe65bc6003c33eba"
format_stamp: "Formatted at 2026-08-07 05:04:45 on dist-test-slave-1x4g"
server_key: "8b001013b327ef76c6e4080f5e6ff806"
server_key_iv: "d6c13651567e3cf478639e3b128e9bcb"
server_key_version: "encryptionkey@0"
I20260807 05:04:45.630231 4106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "5824c2216b7f4f8bbe65bc6003c33eba"
format_stamp: "Formatted at 2026-08-07 05:04:45 on dist-test-slave-1x4g"
server_key: "8b001013b327ef76c6e4080f5e6ff806"
server_key_iv: "d6c13651567e3cf478639e3b128e9bcb"
server_key_version: "encryptionkey@0"
I20260807 05:04:45.633913 4106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:04:45.636260 4121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:45.637478 4106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:45.637621 4106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "5824c2216b7f4f8bbe65bc6003c33eba"
format_stamp: "Formatted at 2026-08-07 05:04:45 on dist-test-slave-1x4g"
server_key: "8b001013b327ef76c6e4080f5e6ff806"
server_key_iv: "d6c13651567e3cf478639e3b128e9bcb"
server_key_version: "encryptionkey@0"
I20260807 05:04:45.637732 4106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:45.650120 4106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:45.652745 4106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:45.652994 4106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:45.660290 4106 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:36257
I20260807 05:04:45.660611 4174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:36257 every 8 connection(s)
I20260807 05:04:45.661273 4106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:45.664057 4175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:45.667732 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4106
I20260807 05:04:45.667873 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:45.668164 1095 external_mini_cluster.cc:1428] Setting key a12a3a39990dc55cecce22257445d22c
I20260807 05:04:45.669960 4175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba: Bootstrap starting.
I20260807 05:04:45.672410 4175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:45.673207 4175 log.cc:824] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:45.675282 4175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba: No bootstrap required, opened a new log
Aug 07 05:04:45 dist-test-slave-1x4g krb5kdc[4093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:45.679473 4175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36257 } }
I20260807 05:04:45.679723 4175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:45.679791 4175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5824c2216b7f4f8bbe65bc6003c33eba, State: Initialized, Role: FOLLOWER
I20260807 05:04:45.680244 4175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36257 } }
I20260807 05:04:45.680444 4175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:45.680528 4175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:45.680611 4175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:45.681634 4175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36257 } }
I20260807 05:04:45.681985 4175 leader_election.cc:302] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5824c2216b7f4f8bbe65bc6003c33eba; no voters:
I20260807 05:04:45.682320 4175 leader_election.cc:288] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:45.682407 4180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:45.682600 4180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [term 1 LEADER]: Becoming Leader. State: Replica: 5824c2216b7f4f8bbe65bc6003c33eba, State: Running, Role: LEADER
I20260807 05:04:45.682734 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:45.669443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.1.17.254:36257)
0807 05:04:45.669841 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:45.669856 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:45.669891 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:04:45.670599 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:45.670617 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:45.670931 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:45.671196 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:45.672084 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:45.673186 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:45.674290 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:45.674747 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:45.674862 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:45.677804 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:45.677810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:45.677822 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:45.677848 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:45.679566 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:45.680082 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:45.680090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:45.680096 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:45.680217 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:45.680502 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:45.680505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:45.680507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:45.680981 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:45.681136 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:45.681444 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 05:04:45.681751 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":119,"threads_started":1}
I20260807 05:04:45.682910 4180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36257 } }
I20260807 05:04:45.683596 4175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:45.684078 4180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36257 } } }
I20260807 05:04:45.684197 4180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:45.685974 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:04:45.686120 4184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:45.685460 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5824c2216b7f4f8bbe65bc6003c33eba. Latest consensus state: current_term: 1 leader_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5824c2216b7f4f8bbe65bc6003c33eba" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36257 } } }
I20260807 05:04:45.686390 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:45.689253 4184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:45.695590 4184 catalog_manager.cc:1347] Generated new cluster ID: 46efccf080704df0b046db6821ce54b9
I20260807 05:04:45.695657 4184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:45.704921 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:45.700201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.1.17.254:36257)
0807 05:04:45.700311 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:45.700314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:45.700558 (+ 244us) server_negotiation.cc:407] Connection header received
0807 05:04:45.700733 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:45.700736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:45.700840 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:45.700952 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:45.701446 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:45.701992 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:45.702853 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:45.703109 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:45.703216 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:45.704063 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:45.704067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:45.704070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:45.704473 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:45.704500 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:45.704515 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:45.704613 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:45.704697 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 05:04:45.704757 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:04:45.709075 4184 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:45.710171 4184 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:45.717309 4184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5824c2216b7f4f8bbe65bc6003c33eba: Generated new TSK 0
I20260807 05:04:45.717916 4184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:04:45.727957 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4106
2026-08-07T05:04:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1419 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:45 dist-test-slave-1x4g krb5kdc[4209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:45 dist-test-slave-1x4g krb5kdc[4209](info): set up 2 sockets
Aug 07 05:04:45 dist-test-slave-1x4g krb5kdc[4209](info): commencing operation
krb5kdc: starting...
W20260807 05:04:46.768482 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:46 dist-test-slave-1x4g krb5kdc[4209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:46.898087 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:42969
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:42969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:47.010878 4222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:47.011142 4222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:47.011180 4222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:47.015318 4222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:47.015375 4222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:47.015407 4222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:47.015424 4222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:47.015460 4222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:47.020578 4222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:42969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:47.021621 4222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:47.022651 4222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:47.031462 4228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:47.031421 4230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:47.031446 4227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:47.031724 4222 server_base.cc:1034] running on GCE node
I20260807 05:04:47.032383 4222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:47.033546 4222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:47.034736 4222 hybrid_clock.cc:648] HybridClock initialized: now 1786079087034703 us; error 44 us; skew 500 ppm
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:47.037501 4222 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:47.038401 4222 webserver.cc:460] Webserver started at http://127.1.17.254:43263/ using document root <none> and password file <none>
I20260807 05:04:47.039003 4222 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:47.039095 4222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:47.039326 4222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:47.041431 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "cd24c553463f4bdcba5e418bcdb12a93"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "748f06a37289bd8bb110e9d5181466e9"
server_key_iv: "6ec91015e55839ed811cd8a11daa97ed"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.042003 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "cd24c553463f4bdcba5e418bcdb12a93"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "748f06a37289bd8bb110e9d5181466e9"
server_key_iv: "6ec91015e55839ed811cd8a11daa97ed"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.045738 4222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:04:47.048087 4238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.049369 4222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:04:47.049532 4222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "cd24c553463f4bdcba5e418bcdb12a93"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "748f06a37289bd8bb110e9d5181466e9"
server_key_iv: "6ec91015e55839ed811cd8a11daa97ed"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.049650 4222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:47.079284 4222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:47.082391 4222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:47.082616 4222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:47.090161 4222 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:42969
I20260807 05:04:47.090456 4290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:42969 every 8 connection(s)
I20260807 05:04:47.091195 4222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:47.094048 4291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:47.094758 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4222
I20260807 05:04:47.094892 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:47.095227 1095 external_mini_cluster.cc:1428] Setting key 5ea52c8958a397a19b3ac3ff323e4cc3
I20260807 05:04:47.101058 4291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93: Bootstrap starting.
I20260807 05:04:47.103408 4291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93: Neither blocks nor log segments found. Creating new log.
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:47.104234 4291 log.cc:824] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:47.106783 4291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93: No bootstrap required, opened a new log
I20260807 05:04:47.110030 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.096646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.1.17.254:42969)
0807 05:04:47.097107 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:47.097122 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:47.097158 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:04:47.097844 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:47.097861 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:47.098196 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:47.098546 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:47.099385 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.100387 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.101372 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.101852 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.102029 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.104564 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:47.104570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:47.104584 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:47.104611 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:47.106625 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.107197 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.107204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.107210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.107351 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.107623 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.107627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.107628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.108158 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:47.108350 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:47.108604 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 05:04:47.108940 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":131,"threads_started":1}
I20260807 05:04:47.111544 4291 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd24c553463f4bdcba5e418bcdb12a93" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42969 } }
I20260807 05:04:47.111756 4291 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:47.111824 4291 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd24c553463f4bdcba5e418bcdb12a93, State: Initialized, Role: FOLLOWER
I20260807 05:04:47.112291 4291 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd24c553463f4bdcba5e418bcdb12a93" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42969 } }
I20260807 05:04:47.112452 4291 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:47.112530 4291 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:47.112627 4291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:47.113622 4291 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd24c553463f4bdcba5e418bcdb12a93" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42969 } }
I20260807 05:04:47.113989 4291 leader_election.cc:302] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd24c553463f4bdcba5e418bcdb12a93; no voters:
I20260807 05:04:47.114313 4291 leader_election.cc:288] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:47.114446 4296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:47.114693 4296 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [term 1 LEADER]: Becoming Leader. State: Replica: cd24c553463f4bdcba5e418bcdb12a93, State: Running, Role: LEADER
I20260807 05:04:47.115093 4296 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd24c553463f4bdcba5e418bcdb12a93" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42969 } }
I20260807 05:04:47.115643 4291 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:47.116781 4298 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd24c553463f4bdcba5e418bcdb12a93. Latest consensus state: current_term: 1 leader_uuid: "cd24c553463f4bdcba5e418bcdb12a93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd24c553463f4bdcba5e418bcdb12a93" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42969 } } }
I20260807 05:04:47.116968 4298 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:47.117137 4297 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd24c553463f4bdcba5e418bcdb12a93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd24c553463f4bdcba5e418bcdb12a93" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42969 } } }
I20260807 05:04:47.117244 4297 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:47.117520 4305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:47.120752 4305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:47.125986 4305 catalog_manager.cc:1347] Generated new cluster ID: 0f94c1976b9c4ab3a2ee039ce8fbab03
I20260807 05:04:47.126052 4305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:47.131448 4305 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:47.132357 4305 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:47.146237 4305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd24c553463f4bdcba5e418bcdb12a93: Generated new TSK 0
I20260807 05:04:47.146932 4305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:47.186085 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42969
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:47.300148 4317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:47.300388 4317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:47.300431 4317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:47.304353 4317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:47.304406 4317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:47.304462 4317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:47.309497 4317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:47.310432 4317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:47.311383 4317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:47.321018 4323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:47.321089 4317 server_base.cc:1034] running on GCE node
W20260807 05:04:47.321039 4325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:47.321040 4322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:47.321691 4317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:47.322364 4317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:47.323537 4317 hybrid_clock.cc:648] HybridClock initialized: now 1786079087323507 us; error 54 us; skew 500 ppm
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:47.326300 4317 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:47.327091 4317 webserver.cc:460] Webserver started at http://127.1.17.193:43377/ using document root <none> and password file <none>
I20260807 05:04:47.327598 4317 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:47.327658 4317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:47.327818 4317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:47.329834 4317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "b0c39c71eb074980831fbd82bff445a5"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "54b0da2a824c20d9b6c4cce3e22b2ee7"
server_key_iv: "d5ef200f4d34c5cbeb756da46937b87a"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.330343 4317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "b0c39c71eb074980831fbd82bff445a5"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "54b0da2a824c20d9b6c4cce3e22b2ee7"
server_key_iv: "d5ef200f4d34c5cbeb756da46937b87a"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.334007 4317 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:04:47.336387 4332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.337592 4317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:47.337728 4317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "b0c39c71eb074980831fbd82bff445a5"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "54b0da2a824c20d9b6c4cce3e22b2ee7"
server_key_iv: "d5ef200f4d34c5cbeb756da46937b87a"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.337837 4317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:47.347608 4317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:47.350524 4317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:47.350713 4317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:47.351251 4317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:47.352025 4317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:47.352093 4317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.352160 4317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:47.352208 4317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.361115 4317 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:34393
I20260807 05:04:47.361161 4449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:34393 every 8 connection(s)
I20260807 05:04:47.361974 4317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:47.362576 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4317
I20260807 05:04:47.362710 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:47.363037 1095 external_mini_cluster.cc:1428] Setting key 7e9af000a8660af39ceee6c9c80104cd
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:04:47.377858 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.363969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:56821 (local address 127.1.17.254:42969)
0807 05:04:47.364093 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:47.364096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:47.365194 (+ 1098us) server_negotiation.cc:407] Connection header received
0807 05:04:47.367179 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:47.367184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:47.367245 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:47.367339 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:47.368429 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.369333 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.370560 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.371027 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.371153 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.373963 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:47.373966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:47.373969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:47.373992 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:47.375592 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.376041 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.376043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.376045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.376182 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.376506 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.376509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.376510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.376835 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:47.376968 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:47.377558 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 05:04:47.377684 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:04:47.378597 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.364478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:42969 (local address 127.1.17.193:56821)
0807 05:04:47.365024 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:47.365086 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:47.366978 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:47.367498 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:47.367504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:47.367835 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:47.368280 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:47.368293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.369465 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.369468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:47.370432 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:47.370443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.371173 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.371176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:47.371312 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.371955 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:47.371972 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:47.373791 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:47.375698 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:47.375703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:47.375713 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:47.375953 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:47.376294 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:47.376296 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:47.376297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:47.376420 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:47.376985 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:47.376995 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:47.377306 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 05:04:47.377514 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 05:04:47.377822 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":119,"threads_started":1}
I20260807 05:04:47.379967 4450 heartbeater.cc:347] Connected to a master server at 127.1.17.254:42969
I20260807 05:04:47.380223 4450 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:47.380704 4450 heartbeater.cc:511] Master 127.1.17.254:42969 requested a full tablet report, sending...
I20260807 05:04:47.382467 4255 ts_manager.cc:194] Registered new tserver with Master: b0c39c71eb074980831fbd82bff445a5 (127.1.17.193:34393)
I20260807 05:04:47.384158 4255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:56821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:47.395319 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42969
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:47.505183 4456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:47.505422 4456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:47.505462 4456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:47.509447 4456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:47.509505 4456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:47.509559 4456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:47.514608 4456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:47.515643 4456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:47.516803 4456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:47.526429 4463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:47.526469 4465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:47.526429 4462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:47.526670 4456 server_base.cc:1034] running on GCE node
I20260807 05:04:47.527083 4456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:47.527700 4456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:47.528851 4456 hybrid_clock.cc:648] HybridClock initialized: now 1786079087528838 us; error 29 us; skew 500 ppm
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:47.531626 4456 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:47.532464 4456 webserver.cc:460] Webserver started at http://127.1.17.194:44975/ using document root <none> and password file <none>
I20260807 05:04:47.533068 4456 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:47.533159 4456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:47.533380 4456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:47.535362 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "9b6194ee5c7d442d9c71cfdc38125522"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "3c18e2f96baca06965b5a5afe4ee8f86"
server_key_iv: "f2f4c64f6dc4709a454e1c00e153525a"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.535944 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "9b6194ee5c7d442d9c71cfdc38125522"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "3c18e2f96baca06965b5a5afe4ee8f86"
server_key_iv: "f2f4c64f6dc4709a454e1c00e153525a"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.539565 4456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 05:04:47.542131 4472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.543330 4456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:47.543474 4456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "9b6194ee5c7d442d9c71cfdc38125522"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "3c18e2f96baca06965b5a5afe4ee8f86"
server_key_iv: "f2f4c64f6dc4709a454e1c00e153525a"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.543583 4456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:47.558154 4456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:47.561496 4456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:47.561687 4456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:47.562239 4456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:47.563025 4456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:47.563095 4456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.563163 4456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:47.563213 4456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.572134 4456 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:45079
I20260807 05:04:47.572163 4589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:45079 every 8 connection(s)
I20260807 05:04:47.573071 4456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:47.581642 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4456
I20260807 05:04:47.581743 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:47.581990 1095 external_mini_cluster.cc:1428] Setting key 1632c8d341868a434f9f8f85cec4a5ac
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:47.586645 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.574737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:38583 (local address 127.1.17.254:42969)
0807 05:04:47.574861 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:47.574865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:47.575612 (+ 747us) server_negotiation.cc:407] Connection header received
0807 05:04:47.577119 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:47.577122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:47.577186 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:47.577265 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:47.578382 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.578928 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.580052 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.580295 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.580410 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.582836 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:47.582840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:47.582843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:47.582875 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:47.584494 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.585116 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.585120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.585122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.585208 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.585568 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.585571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.585573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.585757 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:47.585869 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:47.586354 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 05:04:47.586477 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:04:47.587402 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.575041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:42969 (local address 127.1.17.194:38583)
0807 05:04:47.575472 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:47.575513 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:47.576891 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:47.577429 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:47.577437 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:47.577774 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:47.578228 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:47.578241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.579061 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.579064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:47.579916 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:47.579925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.580446 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.580450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:47.580622 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.581164 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:47.581180 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:47.582680 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:47.584643 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:47.584649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:47.584663 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:47.585009 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:47.585306 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:47.585310 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:47.585312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:47.585465 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:47.585868 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:47.585876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:47.586113 (+ 237us) client_negotiation.cc:769] Sending connection context
0807 05:04:47.586290 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 05:04:47.586605 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260807 05:04:47.588582 4590 heartbeater.cc:347] Connected to a master server at 127.1.17.254:42969
I20260807 05:04:47.588856 4590 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:47.589469 4590 heartbeater.cc:511] Master 127.1.17.254:42969 requested a full tablet report, sending...
I20260807 05:04:47.590709 4255 ts_manager.cc:194] Registered new tserver with Master: 9b6194ee5c7d442d9c71cfdc38125522 (127.1.17.194:45079)
I20260807 05:04:47.591599 4255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:38583
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:47.611424 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42969
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:47.725677 4595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:47.725910 4595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:47.725956 4595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:47.729974 4595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:47.730027 4595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:47.730082 4595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:47.735136 4595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:47.736084 4595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:47.737126 4595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:47.746891 4600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:47.746927 4603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:47.747085 4595 server_base.cc:1034] running on GCE node
W20260807 05:04:47.746908 4601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:47.747710 4595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:47.748358 4595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:47.749534 4595 hybrid_clock.cc:648] HybridClock initialized: now 1786079087749507 us; error 45 us; skew 500 ppm
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:47.752266 4595 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:47.753228 4595 webserver.cc:460] Webserver started at http://127.1.17.195:41477/ using document root <none> and password file <none>
I20260807 05:04:47.753789 4595 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:47.753881 4595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:47.754092 4595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:47.756036 4595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "1111a9dd32fa4d03bfc3720f2985658f"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "d0fb21bd0957a3b1a316540b6f6709f0"
server_key_iv: "d80c768eff4075b9dbd8f9c952d9a967"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.756572 4595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "1111a9dd32fa4d03bfc3720f2985658f"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "d0fb21bd0957a3b1a316540b6f6709f0"
server_key_iv: "d80c768eff4075b9dbd8f9c952d9a967"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.760120 4595 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:47.762437 4610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.763607 4595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:47.763695 4595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "1111a9dd32fa4d03bfc3720f2985658f"
format_stamp: "Formatted at 2026-08-07 05:04:47 on dist-test-slave-1x4g"
server_key: "d0fb21bd0957a3b1a316540b6f6709f0"
server_key_iv: "d80c768eff4075b9dbd8f9c952d9a967"
server_key_version: "encryptionkey@0"
I20260807 05:04:47.763775 4595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:47.795745 4595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:47.798727 4595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:47.798879 4595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:47.799398 4595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:47.800175 4595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:47.800247 4595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.800298 4595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:47.800312 4595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:47.809414 4595 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:45989
I20260807 05:04:47.809446 4727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:45989 every 8 connection(s)
I20260807 05:04:47.810299 4595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:47.818161 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4595
I20260807 05:04:47.818286 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:47.818579 1095 external_mini_cluster.cc:1428] Setting key fad10b97237d899b893c7e21454d23da
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:47.823529 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.812080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:45511 (local address 127.1.17.254:42969)
0807 05:04:47.812225 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:47.812229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:47.813107 (+ 878us) server_negotiation.cc:407] Connection header received
0807 05:04:47.814446 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:47.814449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:47.814498 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:47.814570 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:47.815726 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.816324 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.817599 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.817815 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.817934 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.820551 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:47.820554 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:47.820557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:47.820583 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:47.821824 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.822278 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.822280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.822282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.822368 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.822653 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.822655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.822656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.822812 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:47.822883 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:47.823301 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 05:04:47.823394 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:04:47.824406 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.812419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:42969 (local address 127.1.17.195:45511)
0807 05:04:47.812949 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:47.812996 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:47.814234 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:47.814698 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:47.814704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:47.815088 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:47.815597 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:47.815611 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.816439 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.816442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:47.817495 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:47.817505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.817991 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.817994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:47.818131 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.818822 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:47.818842 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:47.820393 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:47.821926 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:47.821930 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:47.821940 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:47.822199 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:47.822448 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:47.822450 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:47.822451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:47.822579 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:47.822896 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:47.822901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:47.823111 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 05:04:47.823257 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 05:04:47.823518 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260807 05:04:47.825471 4728 heartbeater.cc:347] Connected to a master server at 127.1.17.254:42969
I20260807 05:04:47.825685 4728 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:47.826125 4728 heartbeater.cc:511] Master 127.1.17.254:42969 requested a full tablet report, sending...
I20260807 05:04:47.827091 4255 ts_manager.cc:194] Registered new tserver with Master: 1111a9dd32fa4d03bfc3720f2985658f (127.1.17.195:45989)
I20260807 05:04:47.827634 4255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:45511
I20260807 05:04:47.832425 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079087, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:47.858666 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:47.850714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.1.17.254:42969)
0807 05:04:47.850843 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:47.850846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:47.851012 (+ 166us) server_negotiation.cc:407] Connection header received
0807 05:04:47.851123 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:47.851125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:47.851167 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:47.851238 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:47.851884 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.852373 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.853336 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:47.853554 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:47.853635 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:47.855967 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:47.855970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:47.855972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:47.855995 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:47.857268 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.857653 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.857655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.857656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.857727 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:47.857952 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:47.857955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:47.857956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:47.858160 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:47.858262 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:47.858445 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 05:04:47.858545 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 05:04:47.859053 4255 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45726
I20260807 05:04:47.859946 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4317
I20260807 05:04:47.869225 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4456
I20260807 05:04:47.877636 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4595
I20260807 05:04:47.883298 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4222
2026-08-07T05:04:47Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2155 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4741](info): set up 2 sockets
Aug 07 05:04:47 dist-test-slave-1x4g krb5kdc[4741](info): commencing operation
krb5kdc: starting...
W20260807 05:04:48.941946 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:49.068569 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44021
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:44021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:49.178056 4755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:49.178337 4755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:49.178411 4755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:49.182205 4755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:49.182296 4755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:49.182356 4755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:49.182410 4755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:49.182451 4755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:49.187592 4755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:44021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:49.188608 4755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:49.189632 4755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:49.198222 4763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:49.198133 4760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:49.198334 4755 server_base.cc:1034] running on GCE node
W20260807 05:04:49.198505 4761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:49.199071 4755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:49.200098 4755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:49.201253 4755 hybrid_clock.cc:648] HybridClock initialized: now 1786079089201225 us; error 40 us; skew 500 ppm
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:49.204128 4755 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:49.205082 4755 webserver.cc:460] Webserver started at http://127.1.17.254:33259/ using document root <none> and password file <none>
I20260807 05:04:49.205649 4755 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:49.205739 4755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:49.205976 4755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:49.207947 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "2a1517cbe43d49a08811375d2679c07f"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "b45d746572611aa8718ff4c046f76d14"
server_key_iv: "981bf5edc8d09d855b7e9e43944d75df"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.208523 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "2a1517cbe43d49a08811375d2679c07f"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "b45d746572611aa8718ff4c046f76d14"
server_key_iv: "981bf5edc8d09d855b7e9e43944d75df"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.212239 4755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:49.214677 4770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.215835 4755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:49.215997 4755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "2a1517cbe43d49a08811375d2679c07f"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "b45d746572611aa8718ff4c046f76d14"
server_key_iv: "981bf5edc8d09d855b7e9e43944d75df"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.216111 4755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:49.232409 4755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:49.235656 4755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:49.235867 4755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:49.242991 4755 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:44021
I20260807 05:04:49.243006 4824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:44021 every 8 connection(s)
I20260807 05:04:49.243887 4755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:49.244655 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4755
I20260807 05:04:49.244761 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:49.245100 1095 external_mini_cluster.cc:1428] Setting key 9e775e4f584b30825ba5deea6cdd473e
I20260807 05:04:49.247128 4825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:49.253794 4825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f: Bootstrap starting.
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:49.256526 4825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:49.257457 4825 log.cc:824] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:49.259699 4825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f: No bootstrap required, opened a new log
I20260807 05:04:49.260419 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:49.246523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.1.17.254:44021)
0807 05:04:49.246962 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:49.246982 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:49.247042 (+ 60us) server_negotiation.cc:407] Connection header received
0807 05:04:49.247908 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:49.247930 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:49.248336 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:49.248694 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:49.249387 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.250918 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.251733 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.252463 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.252640 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:49.255404 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:49.255410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:49.255425 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:49.255464 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:49.257414 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.257876 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.257881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.257887 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.258001 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.258305 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.258307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.258309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.258778 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:49.258969 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:49.259209 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 05:04:49.259508 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260807 05:04:49.263581 4825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1517cbe43d49a08811375d2679c07f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44021 } }
I20260807 05:04:49.263782 4825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:49.263854 4825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1517cbe43d49a08811375d2679c07f, State: Initialized, Role: FOLLOWER
I20260807 05:04:49.264240 4825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1517cbe43d49a08811375d2679c07f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44021 } }
I20260807 05:04:49.264396 4825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:49.264492 4825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:49.264590 4825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:49.265470 4825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1517cbe43d49a08811375d2679c07f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44021 } }
I20260807 05:04:49.265805 4825 leader_election.cc:302] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1517cbe43d49a08811375d2679c07f; no voters:
I20260807 05:04:49.266104 4825 leader_election.cc:288] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:49.266251 4830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:49.266616 4830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [term 1 LEADER]: Becoming Leader. State: Replica: 2a1517cbe43d49a08811375d2679c07f, State: Running, Role: LEADER
I20260807 05:04:49.267017 4830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1517cbe43d49a08811375d2679c07f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44021 } }
I20260807 05:04:49.267380 4825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:49.268639 4831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a1517cbe43d49a08811375d2679c07f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1517cbe43d49a08811375d2679c07f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44021 } } }
I20260807 05:04:49.268934 4832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a1517cbe43d49a08811375d2679c07f. Latest consensus state: current_term: 1 leader_uuid: "2a1517cbe43d49a08811375d2679c07f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1517cbe43d49a08811375d2679c07f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44021 } } }
I20260807 05:04:49.269053 4832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:49.268766 4831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f [sys.catalog]: This master's current role is: LEADER
W20260807 05:04:49.272264 4845 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:04:49.272359 4845 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:04:49.272444 4843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:49.273231 4843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:49.278319 4843 catalog_manager.cc:1347] Generated new cluster ID: 6df412b79e1443809ac557dff069ad18
I20260807 05:04:49.278383 4843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:49.305523 4843 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:49.306474 4843 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:49.313467 4843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a1517cbe43d49a08811375d2679c07f: Generated new TSK 0
I20260807 05:04:49.314205 4843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:49.343618 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44021
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:49.454695 4851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:49.454989 4851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:49.455072 4851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:49.459110 4851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:49.459220 4851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:49.459318 4851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:49.464349 4851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:49.465471 4851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:49.466528 4851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:49.476301 4858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:49.476325 4857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:49.476306 4860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:49.476683 4851 server_base.cc:1034] running on GCE node
I20260807 05:04:49.477120 4851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:49.477736 4851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:49.478897 4851 hybrid_clock.cc:648] HybridClock initialized: now 1786079089478868 us; error 40 us; skew 500 ppm
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:49.481614 4851 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:49.482519 4851 webserver.cc:460] Webserver started at http://127.1.17.193:41891/ using document root <none> and password file <none>
I20260807 05:04:49.483079 4851 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:49.483170 4851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:49.483402 4851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:49.485531 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "c31d6d2c57ce4d89bfc812e7a14c3a85"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "d429cb9f93d73e0ec3d7a4e0bca32b14"
server_key_iv: "f869233a37cd29fa40b8036523ffd6e5"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.486104 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "c31d6d2c57ce4d89bfc812e7a14c3a85"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "d429cb9f93d73e0ec3d7a4e0bca32b14"
server_key_iv: "f869233a37cd29fa40b8036523ffd6e5"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.489858 4851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:04:49.492323 4867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.493639 4851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:49.493775 4851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "c31d6d2c57ce4d89bfc812e7a14c3a85"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "d429cb9f93d73e0ec3d7a4e0bca32b14"
server_key_iv: "f869233a37cd29fa40b8036523ffd6e5"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.493883 4851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:49.503940 4851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:49.507282 4851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:49.507483 4851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:49.508049 4851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:49.508915 4851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:49.508982 4851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.509047 4851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:49.509092 4851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.518604 4851 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:46557
I20260807 05:04:49.518639 4986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:46557 every 8 connection(s)
I20260807 05:04:49.519570 4851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:49.529867 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4851
I20260807 05:04:49.530014 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:49.530369 1095 external_mini_cluster.cc:1428] Setting key fe03e1b5b9fd1424e9fd8eca9689013e
I20260807 05:04:49.533254 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:49.521370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:44585 (local address 127.1.17.254:44021)
0807 05:04:49.521517 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:49.521522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:49.522376 (+ 854us) server_negotiation.cc:407] Connection header received
0807 05:04:49.523679 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:49.523682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:49.523728 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:49.523803 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:49.524955 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.525505 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.526610 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.526848 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.526923 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:49.529244 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:49.529247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:49.529249 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:49.529272 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:49.530856 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.531552 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.531555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.531557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.531656 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.532107 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.532110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.532111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.532290 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:49.532368 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:49.533032 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 05:04:49.533136 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:04:49.534396 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:49.521790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44021 (local address 127.1.17.193:44585)
0807 05:04:49.522229 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:49.522270 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:49.523486 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:49.523976 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:49.523983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:49.524315 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:49.524819 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:49.524834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.525614 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.525617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:49.526502 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:49.526511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.526981 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.526984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:49.527122 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:49.527624 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:49.527639 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:49.529095 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:49.531179 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:49.531183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:49.531195 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:49.531450 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:49.531786 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:49.531790 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:49.531791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:49.532000 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:49.532405 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:49.532414 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:49.532731 (+ 317us) client_negotiation.cc:769] Sending connection context
0807 05:04:49.532991 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 05:04:49.533351 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
I20260807 05:04:49.535594 4987 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44021
I20260807 05:04:49.535837 4987 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:49.536408 4987 heartbeater.cc:511] Master 127.1.17.254:44021 requested a full tablet report, sending...
I20260807 05:04:49.538062 4787 ts_manager.cc:194] Registered new tserver with Master: c31d6d2c57ce4d89bfc812e7a14c3a85 (127.1.17.193:46557)
I20260807 05:04:49.539204 4787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:44585
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:49.559729 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44021
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:49.672758 4992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:49.673120 4992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:49.673199 4992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:49.677356 4992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:49.677433 4992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:49.677534 4992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:49.683264 4992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:49.684226 4992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:49.685268 4992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:49.695070 5000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:49.695106 4998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:49.695106 4997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:49.695271 4992 server_base.cc:1034] running on GCE node
I20260807 05:04:49.695802 4992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:49.696362 4992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:49.697546 4992 hybrid_clock.cc:648] HybridClock initialized: now 1786079089697514 us; error 46 us; skew 500 ppm
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:49.700368 4992 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:49.701277 4992 webserver.cc:460] Webserver started at http://127.1.17.194:33415/ using document root <none> and password file <none>
I20260807 05:04:49.701782 4992 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:49.701843 4992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:49.702014 4992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:49.703966 4992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "0770b1153fae49dab1545b78742345a4"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "fb02accb3fc5a3c75fead05b88ed735b"
server_key_iv: "91b67b2572bc6e947e97ae11579cb5d9"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.704468 4992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "0770b1153fae49dab1545b78742345a4"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "fb02accb3fc5a3c75fead05b88ed735b"
server_key_iv: "91b67b2572bc6e947e97ae11579cb5d9"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.708361 4992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 05:04:49.710803 5007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.712105 4992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:04:49.712242 4992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "0770b1153fae49dab1545b78742345a4"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "fb02accb3fc5a3c75fead05b88ed735b"
server_key_iv: "91b67b2572bc6e947e97ae11579cb5d9"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.712354 4992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:49.749192 4992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:49.752329 4992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:49.752549 4992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:49.753240 4992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:49.754117 4992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:49.754189 4992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.754263 4992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:49.754315 4992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.763635 4992 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:42257
I20260807 05:04:49.763670 5124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:42257 every 8 connection(s)
I20260807 05:04:49.764569 4992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:49.766505 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 4992
I20260807 05:04:49.766623 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:49.766981 1095 external_mini_cluster.cc:1428] Setting key d12886e115ef89ed75c0fa71a2c75971
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:04:49.780018 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:49.766561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:51131 (local address 127.1.17.254:44021)
0807 05:04:49.766720 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:49.766727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:49.767929 (+ 1202us) server_negotiation.cc:407] Connection header received
0807 05:04:49.770003 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:49.770005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:49.770046 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:49.770107 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:49.771604 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.772131 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.773406 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.773634 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.773737 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:49.776378 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:49.776381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:49.776382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:49.776404 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:49.777979 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.778492 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.778496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.778497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.778580 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.778929 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.778934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.778939 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.779120 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:49.779229 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:49.779727 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 05:04:49.779848 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 05:04:49.780817 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:49.766995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44021 (local address 127.1.17.194:51131)
0807 05:04:49.767763 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:49.767821 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:49.769753 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:49.770312 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:49.770323 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:49.770890 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:49.771459 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:49.771474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.772274 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.772278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:49.773270 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:49.773280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.773787 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.773791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:49.773970 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:49.774654 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:49.774670 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:49.776218 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:49.778092 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:49.778097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:49.778111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:49.778384 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:49.778682 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:49.778685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:49.778687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:49.778831 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:49.779241 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:49.779249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:49.779502 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 05:04:49.779683 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 05:04:49.779997 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":182,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
I20260807 05:04:49.782054 5125 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44021
I20260807 05:04:49.782306 5125 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:49.782989 5125 heartbeater.cc:511] Master 127.1.17.254:44021 requested a full tablet report, sending...
I20260807 05:04:49.784189 4787 ts_manager.cc:194] Registered new tserver with Master: 0770b1153fae49dab1545b78742345a4 (127.1.17.194:42257)
I20260807 05:04:49.784976 4787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:51131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:49.798062 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44021
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:49.909960 5130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:49.910221 5130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:49.910297 5130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:49.914284 5130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:49.914362 5130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:49.914455 5130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:49.919680 5130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:49.920753 5130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:49.921784 5130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:49.931051 5135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:49.931061 5138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:49.931058 5136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:49.931377 5130 server_base.cc:1034] running on GCE node
I20260807 05:04:49.931800 5130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:49.932410 5130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:49.933604 5130 hybrid_clock.cc:648] HybridClock initialized: now 1786079089933588 us; error 23 us; skew 500 ppm
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:49.936412 5130 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:49.937371 5130 webserver.cc:460] Webserver started at http://127.1.17.195:33929/ using document root <none> and password file <none>
I20260807 05:04:49.937932 5130 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:49.938022 5130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:49.938234 5130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:49.940179 5130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "c6e9a92f863d440ba4f1e3a4f98aaab4"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "ea014cb36aac132746eff131bf5158e9"
server_key_iv: "b4efa2138afe264b9e6a495a4bbcd90e"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.940725 5130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "c6e9a92f863d440ba4f1e3a4f98aaab4"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "ea014cb36aac132746eff131bf5158e9"
server_key_iv: "b4efa2138afe264b9e6a495a4bbcd90e"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.944406 5130 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:04:49.946743 5145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.947890 5130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:49.948055 5130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "c6e9a92f863d440ba4f1e3a4f98aaab4"
format_stamp: "Formatted at 2026-08-07 05:04:49 on dist-test-slave-1x4g"
server_key: "ea014cb36aac132746eff131bf5158e9"
server_key_iv: "b4efa2138afe264b9e6a495a4bbcd90e"
server_key_version: "encryptionkey@0"
I20260807 05:04:49.948166 5130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:49.966833 5130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:49.970355 5130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:49.970561 5130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:49.971143 5130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:49.971980 5130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:49.972051 5130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.972123 5130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:49.972177 5130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:49.981544 5130 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:35415
I20260807 05:04:49.981565 5263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:35415 every 8 connection(s)
I20260807 05:04:49.982456 5130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:49.984476 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 5130
I20260807 05:04:49.984619 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:49.984980 1095 external_mini_cluster.cc:1428] Setting key c02b66994086390d6cc5db1b957b72c3
Aug 07 05:04:49 dist-test-slave-1x4g krb5kdc[4741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079089, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:49.997473 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:49.984318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:40895 (local address 127.1.17.254:44021)
0807 05:04:49.984505 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:49.984510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:49.986222 (+ 1712us) server_negotiation.cc:407] Connection header received
0807 05:04:49.987620 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:49.987624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:49.987677 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:49.987754 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:49.988995 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.989568 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.990759 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.991037 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.991163 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:49.993762 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:49.993772 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:49.993775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:49.993799 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:49.995272 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.995801 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.995806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.995807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.995888 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:49.996246 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:49.996250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:49.996251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:49.996435 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:49.996538 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:49.997115 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 05:04:49.997309 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 05:04:49.998461 5266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:49.984879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44021 (local address 127.1.17.195:40895)
0807 05:04:49.986078 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:49.986121 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:49.987409 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:49.987923 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:49.987930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:49.988308 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:49.988856 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:49.988873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.989703 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.989706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:49.990610 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:49.990620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:49.991204 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:49.991208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:49.991397 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:49.992043 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:49.992059 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:49.993593 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:49.995403 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:49.995408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:49.995423 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:49.995684 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:49.995985 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:49.995988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:49.995990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:49.996142 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:49.996551 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:49.996559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:49.996870 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 05:04:49.997057 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 05:04:49.997389 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"thread_start_us":161,"threads_started":1}
I20260807 05:04:49.999500 5264 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44021
I20260807 05:04:49.999735 5264 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:50.000229 5264 heartbeater.cc:511] Master 127.1.17.254:44021 requested a full tablet report, sending...
I20260807 05:04:50.001307 4787 ts_manager.cc:194] Registered new tserver with Master: c6e9a92f863d440ba4f1e3a4f98aaab4 (127.1.17.195:35415)
I20260807 05:04:50.001866 4787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:40895
I20260807 05:04:50.010531 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:04:50 dist-test-slave-1x4g krb5kdc[4741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:04:50 dist-test-slave-1x4g krb5kdc[4741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:50.037580 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:50.027730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.1.17.193:46557)
0807 05:04:50.028049 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:50.028065 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:50.028091 (+ 26us) server_negotiation.cc:407] Connection header received
0807 05:04:50.028222 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:50.028227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:50.028420 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:50.028572 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:50.029318 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:50.030096 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:50.030928 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:50.031300 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:50.031428 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:50.033800 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:50.033809 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:50.033821 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:50.033844 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:50.035626 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:50.036110 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:50.036116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:50.036117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:50.036199 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:50.036495 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:50.036506 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:50.036512 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:50.036767 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:50.037052 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:50.037236 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 05:04:50.037394 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
Aug 07 05:04:50 dist-test-slave-1x4g krb5kdc[4741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:50.048645 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:50.040521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.1.17.254:44021)
0807 05:04:50.040679 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:50.040683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:50.040701 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:04:50.040814 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:50.040818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:50.040876 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:50.040954 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:50.041492 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:50.041998 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:50.042720 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:50.042935 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:50.043018 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:50.045393 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:50.045397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:50.045399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:50.045423 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:50.046830 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:50.047273 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:50.047277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:50.047279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:50.047377 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:50.047678 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:50.047682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:50.047684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:50.047912 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:50.048058 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:50.048337 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 05:04:50.048484 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 05:04:50.049077 4787 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46596
I20260807 05:04:50.049757 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4851
I20260807 05:04:50.057945 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4992
I20260807 05:04:50.064210 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 5130
I20260807 05:04:50.070647 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 4755
2026-08-07T05:04:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2187 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:50 dist-test-slave-1x4g krb5kdc[5276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:50 dist-test-slave-1x4g krb5kdc[5276](info): set up 2 sockets
Aug 07 05:04:50 dist-test-slave-1x4g krb5kdc[5276](info): commencing operation
krb5kdc: starting...
W20260807 05:04:51.134352 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:51 dist-test-slave-1x4g krb5kdc[5276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:51.259740 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:41831
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:41831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:51.369331 5289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:51.369557 5289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:51.369596 5289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:51.374001 5289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:51.374047 5289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:51.374064 5289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:51.374099 5289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:51.374130 5289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:51.379231 5289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:41831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:41831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:51.380210 5289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:51.381245 5289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:51.389766 5298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:51.389768 5296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:51.389914 5295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:51.390132 5289 server_base.cc:1034] running on GCE node
I20260807 05:04:51.390645 5289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:51.391705 5289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:51.392844 5289 hybrid_clock.cc:648] HybridClock initialized: now 1786079091392817 us; error 45 us; skew 500 ppm
Aug 07 05:04:51 dist-test-slave-1x4g krb5kdc[5276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079091, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:51.400637 5289 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:51.401568 5289 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:51.401682 5289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:51.401939 5289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:51.404039 5289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "47487884f59849d8aa46e142046bf353"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "667c372f0bd450232609b7e65ba32b28"
server_key_iv: "2019044b7c3d40ccd924cab657a6b438"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.404633 5289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "47487884f59849d8aa46e142046bf353"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "667c372f0bd450232609b7e65ba32b28"
server_key_iv: "2019044b7c3d40ccd924cab657a6b438"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.408351 5289 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:04:51.410913 5304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:51.412132 5289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:04:51.412279 5289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "47487884f59849d8aa46e142046bf353"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "667c372f0bd450232609b7e65ba32b28"
server_key_iv: "2019044b7c3d40ccd924cab657a6b438"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.412389 5289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:51.429044 5289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:51.431695 5289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:51.431943 5289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:51.439308 5289 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:41831
I20260807 05:04:51.439584 5357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:41831 every 8 connection(s)
I20260807 05:04:51.439965 5289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:51.442618 5358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:51.446009 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 5289
I20260807 05:04:51.446106 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:51.446382 1095 external_mini_cluster.cc:1428] Setting key 4c561d0521fe7a090c239dcc71890102
I20260807 05:04:51.448426 5358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353: Bootstrap starting.
I20260807 05:04:51.450497 5358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:51.451210 5358 log.cc:824] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:51.453004 5358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353: No bootstrap required, opened a new log
Aug 07 05:04:51 dist-test-slave-1x4g krb5kdc[5276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:51.457136 5358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47487884f59849d8aa46e142046bf353" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 41831 } }
I20260807 05:04:51.457336 5358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:51.457413 5358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47487884f59849d8aa46e142046bf353, State: Initialized, Role: FOLLOWER
I20260807 05:04:51.457850 5358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47487884f59849d8aa46e142046bf353" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 41831 } }
I20260807 05:04:51.457962 5358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:51.458035 5358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:51.458128 5358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:51.458931 5358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47487884f59849d8aa46e142046bf353" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 41831 } }
I20260807 05:04:51.459295 5358 leader_election.cc:302] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47487884f59849d8aa46e142046bf353; no voters:
I20260807 05:04:51.459642 5358 leader_election.cc:288] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:51.459748 5363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:51.459965 5363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [term 1 LEADER]: Becoming Leader. State: Replica: 47487884f59849d8aa46e142046bf353, State: Running, Role: LEADER
I20260807 05:04:51.460335 5363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47487884f59849d8aa46e142046bf353" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 41831 } }
I20260807 05:04:51.460744 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:51.447761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.1.17.254:41831)
0807 05:04:51.448190 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:51.448207 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:51.448241 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:04:51.448920 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:51.448938 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:51.449234 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:51.449573 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:51.450374 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.451419 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.452262 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.452719 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.452932 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:51.455603 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:51.455609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:51.455621 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:51.455646 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:51.457375 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:51.457887 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:51.457892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:51.457904 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:51.458061 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:51.458333 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:51.458337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:51.458338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:51.458729 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:51.458926 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:51.459250 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 05:04:51.459571 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":123,"threads_started":1}
I20260807 05:04:51.460991 5358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:51.461557 5366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47487884f59849d8aa46e142046bf353. Latest consensus state: current_term: 1 leader_uuid: "47487884f59849d8aa46e142046bf353" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47487884f59849d8aa46e142046bf353" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 41831 } } }
I20260807 05:04:51.461764 5366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:51.462008 5365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47487884f59849d8aa46e142046bf353" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47487884f59849d8aa46e142046bf353" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 41831 } } }
I20260807 05:04:51.462108 5365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:51.462994 5372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:51.465778 5372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:51.471098 5372 catalog_manager.cc:1347] Generated new cluster ID: d7d2a88ede754cc7a27c3b4ad35479c4
I20260807 05:04:51.471166 5372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:51.490139 5372 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:51.491171 5372 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:51.500360 5372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353: Generated new TSK 0
I20260807 05:04:51.501276 5372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:51.540043 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:41831
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:51.673704 5385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:51.673957 5385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:51.674003 5385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:51.677919 5385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:51.677968 5385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:51.678025 5385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:51.683173 5385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:41831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:51.684235 5385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:51.685310 5385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:51.695369 5390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:51.695614 5393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:51.695647 5391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:51.695780 5385 server_base.cc:1034] running on GCE node
I20260807 05:04:51.696339 5385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:51.697189 5385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:51.698401 5385 hybrid_clock.cc:648] HybridClock initialized: now 1786079091698381 us; error 29 us; skew 500 ppm
Aug 07 05:04:51 dist-test-slave-1x4g krb5kdc[5276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079091, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:51.702510 5385 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:51.703470 5385 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:51.703570 5385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:51.703841 5385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:51.706642 5385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "357bd6a6e8da43f381400aefc30de991"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "845f4e74e4eaace97d4c198fc308c705"
server_key_iv: "648e8aeed701006b5f202028230b5293"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.707420 5385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "357bd6a6e8da43f381400aefc30de991"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "845f4e74e4eaace97d4c198fc308c705"
server_key_iv: "648e8aeed701006b5f202028230b5293"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.712481 5385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 05:04:51.715746 5399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:51.717453 5385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 05:04:51.717588 5385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "357bd6a6e8da43f381400aefc30de991"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "845f4e74e4eaace97d4c198fc308c705"
server_key_iv: "648e8aeed701006b5f202028230b5293"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.717721 5385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:51.746418 5385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:51.749457 5385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:51.749606 5385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:51.750149 5385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:51.750928 5385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:51.750986 5385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:51.751027 5385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:51.751042 5385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:51.760070 5385 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:46047
I20260807 05:04:51.760085 5513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:46047 every 8 connection(s)
I20260807 05:04:51.760658 5385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:51.768311 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 5385
I20260807 05:04:51.768448 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:51.768839 1095 external_mini_cluster.cc:1428] Setting key ae75645ecec086c3576633a5e922ed2f
Aug 07 05:04:51 dist-test-slave-1x4g krb5kdc[5276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079091, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:51.775074 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:51.762385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:56507 (local address 127.1.17.254:41831)
0807 05:04:51.762518 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:51.762521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:51.763345 (+ 824us) server_negotiation.cc:407] Connection header received
0807 05:04:51.764700 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:51.764704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:51.764755 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:51.764909 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:51.766049 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.766604 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.767761 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.768038 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.768136 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:51.771096 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:51.771099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:51.771102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:51.771134 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:51.772707 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:51.773306 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:51.773311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:51.773313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:51.773414 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:51.773779 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:51.773785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:51.773788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:51.774054 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:51.774168 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:51.774682 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 05:04:51.774835 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:04:51.775910 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:51.762735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:41831 (local address 127.1.17.193:56507)
0807 05:04:51.763168 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:51.763211 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:51.764469 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:51.765059 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:51.765066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:51.765432 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:51.765910 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:51.765923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.766720 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.766723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:51.767619 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:51.767628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.768241 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.768247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:51.768493 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:51.769234 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:51.769255 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:51.770900 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:51.772901 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:51.772906 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:51.772921 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:51.773201 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:51.773533 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:51.773535 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:51.773537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:51.773680 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:51.774163 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:51.774168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:51.774423 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 05:04:51.774596 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 05:04:51.774864 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
I20260807 05:04:51.777343 5514 heartbeater.cc:347] Connected to a master server at 127.1.17.254:41831
I20260807 05:04:51.777657 5514 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:51.778255 5514 heartbeater.cc:511] Master 127.1.17.254:41831 requested a full tablet report, sending...
I20260807 05:04:51.779928 5321 ts_manager.cc:194] Registered new tserver with Master: 357bd6a6e8da43f381400aefc30de991 (127.1.17.193:46047)
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:04:51.781256 5321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:56507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:51.799249 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:41831
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:51.913705 5519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:51.913949 5519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:51.914019 5519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:51.918167 5519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:51.918241 5519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:51.918327 5519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:51.923427 5519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:41831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:51.924469 5519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:51.925518 5519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:51.934465 5528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:51.934438 5524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:51.934458 5525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:51.934808 5519 server_base.cc:1034] running on GCE node
I20260807 05:04:51.935200 5519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:51.935801 5519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:51.936972 5519 hybrid_clock.cc:648] HybridClock initialized: now 1786079091936961 us; error 29 us; skew 500 ppm
Aug 07 05:04:51 dist-test-slave-1x4g krb5kdc[5276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079091, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:51.939672 5519 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:51.940379 5519 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:51.940470 5519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:51.940703 5519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:51.942649 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "d770563b176a48f9b16ffd8540f3f20e"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "186cdc60708f53efda4df00af1134594"
server_key_iv: "a5f10d470ac43e0690eb6ecdeba663f6"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.943195 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "d770563b176a48f9b16ffd8540f3f20e"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "186cdc60708f53efda4df00af1134594"
server_key_iv: "a5f10d470ac43e0690eb6ecdeba663f6"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.946820 5519 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 05:04:51.949069 5534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:51.950125 5519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:04:51.950253 5519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "d770563b176a48f9b16ffd8540f3f20e"
format_stamp: "Formatted at 2026-08-07 05:04:51 on dist-test-slave-1x4g"
server_key: "186cdc60708f53efda4df00af1134594"
server_key_iv: "a5f10d470ac43e0690eb6ecdeba663f6"
server_key_version: "encryptionkey@0"
I20260807 05:04:51.950356 5519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:51.960376 5519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:51.963825 5519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:51.964053 5519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:51.964664 5519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:51.965569 5519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:51.965644 5519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:51.965715 5519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:51.965766 5519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:51.975063 5519 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:34735
I20260807 05:04:51.975090 5653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:34735 every 8 connection(s)
I20260807 05:04:51.975695 5519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:51.985661 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 5519
Aug 07 05:04:51 dist-test-slave-1x4g krb5kdc[5276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079091, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:51.985810 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:51.986096 1095 external_mini_cluster.cc:1428] Setting key 3246f64a5aa579c5f067da20db396fbe
I20260807 05:04:51.990445 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:51.977445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:33833 (local address 127.1.17.254:41831)
0807 05:04:51.977584 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:51.977588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:51.978360 (+ 772us) server_negotiation.cc:407] Connection header received
0807 05:04:51.979757 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:51.979761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:51.979821 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:51.979930 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:51.981358 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.981992 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.983357 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.983615 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.983740 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:51.986475 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:51.986480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:51.986483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:51.986516 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:51.988258 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:51.988760 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:51.988767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:51.988772 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:51.988920 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:51.989269 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:51.989276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:51.989281 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:51.989548 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:51.989660 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:51.990137 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 05:04:51.990272 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:04:51.991262 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:51.977747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:41831 (local address 127.1.17.194:33833)
0807 05:04:51.978212 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:51.978254 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:51.979546 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:51.980124 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:51.980131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:51.980598 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:51.981173 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:51.981187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.982160 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.982164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:51.983203 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:51.983213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:51.983784 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:51.983788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:51.983975 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:51.984623 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:51.984641 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:51.986266 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:51.988383 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:51.988387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:51.988398 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:51.988657 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:51.989021 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:51.989023 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:51.989025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:51.989171 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:51.989659 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:51.989664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:51.989874 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 05:04:51.990073 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 05:04:51.990382 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":127,"threads_started":1}
I20260807 05:04:51.992434 5654 heartbeater.cc:347] Connected to a master server at 127.1.17.254:41831
I20260807 05:04:51.992722 5654 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:51.993343 5654 heartbeater.cc:511] Master 127.1.17.254:41831 requested a full tablet report, sending...
I20260807 05:04:51.994544 5321 ts_manager.cc:194] Registered new tserver with Master: d770563b176a48f9b16ffd8540f3f20e (127.1.17.194:34735)
I20260807 05:04:51.995522 5321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:33833
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:52.016018 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:41831
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:52.128459 5659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:52.128708 5659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:52.128751 5659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:52.132731 5659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:52.132794 5659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:52.132848 5659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:52.138458 5659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:41831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:52.139426 5659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:52.140362 5659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:52.149946 5664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:52.149902 5665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:52.149884 5667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:52.150012 5659 server_base.cc:1034] running on GCE node
I20260807 05:04:52.150588 5659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:52.151249 5659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:52.152408 5659 hybrid_clock.cc:648] HybridClock initialized: now 1786079092152378 us; error 50 us; skew 500 ppm
Aug 07 05:04:52 dist-test-slave-1x4g krb5kdc[5276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079092, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:52.155403 5659 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:52.156190 5659 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:52.156284 5659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:52.156519 5659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:52.158618 5659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "e576c80fab0342dca6c5f97c1aa8fb1d"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "76206ac592f999f08eec590f9ed746f5"
server_key_iv: "6e273361c50bb9b12abbc470ffa9424a"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.159176 5659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "e576c80fab0342dca6c5f97c1aa8fb1d"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "76206ac592f999f08eec590f9ed746f5"
server_key_iv: "6e273361c50bb9b12abbc470ffa9424a"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.163082 5659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 05:04:52.165495 5673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:52.166738 5659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:04:52.166867 5659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "e576c80fab0342dca6c5f97c1aa8fb1d"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "76206ac592f999f08eec590f9ed746f5"
server_key_iv: "6e273361c50bb9b12abbc470ffa9424a"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.166971 5659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:52.187570 5659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:52.190793 5659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:52.191004 5659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:52.191639 5659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:52.192487 5659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:52.192570 5659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:52.192668 5659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:52.192721 5659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:52.201408 5659 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:33769
I20260807 05:04:52.201432 5790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:33769 every 8 connection(s)
I20260807 05:04:52.202005 5659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:52.203096 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 5659
I20260807 05:04:52.203222 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:52.203521 1095 external_mini_cluster.cc:1428] Setting key 5c0a40efb8d3b3daa4c67325b4fd6cdf
Aug 07 05:04:52 dist-test-slave-1x4g krb5kdc[5276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079092, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:52.215889 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.203865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:55323 (local address 127.1.17.254:41831)
0807 05:04:52.204013 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.204016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.205106 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 05:04:52.206481 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.206484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.206549 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.206617 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:52.207745 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.208267 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.209477 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.209689 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.209769 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.212141 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:52.212145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:52.212148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:52.212178 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:52.213690 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:52.214199 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:52.214203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:52.214205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:52.214286 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:52.214635 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:52.214639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:52.214641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:52.214826 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:52.214941 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.215543 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.215699 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:04:52.216907 5793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.204359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:41831 (local address 127.1.17.195:55323)
0807 05:04:52.204955 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.205010 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:52.206290 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:52.206752 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:52.206758 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:52.207111 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:52.207625 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.207639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.208379 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.208383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.209362 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.209373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.209815 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.209817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.209985 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.210478 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:52.210494 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:52.211985 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:52.213810 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:52.213815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:52.213829 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:52.214088 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:52.214378 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:52.214381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:52.214383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:52.214521 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:52.214964 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:52.214970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:52.215307 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 05:04:52.215502 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 05:04:52.215830 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":185,"threads_started":1}
I20260807 05:04:52.217993 5791 heartbeater.cc:347] Connected to a master server at 127.1.17.254:41831
I20260807 05:04:52.218196 5791 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:52.218628 5791 heartbeater.cc:511] Master 127.1.17.254:41831 requested a full tablet report, sending...
I20260807 05:04:52.219591 5321 ts_manager.cc:194] Registered new tserver with Master: e576c80fab0342dca6c5f97c1aa8fb1d (127.1.17.195:33769)
I20260807 05:04:52.220103 5321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:55323
I20260807 05:04:52.229497 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:04:52.238876 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.231529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.1.17.254:41831)
0807 05:04:52.231679 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.231683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.231826 (+ 143us) server_negotiation.cc:407] Connection header received
0807 05:04:52.231950 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.231953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.231998 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.232064 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:52.232750 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.233317 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.234162 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.234366 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.234438 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.235705 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:52.235708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:52.235710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:52.235732 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:52.237326 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:52.237709 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:52.237712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:52.237713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:52.237786 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:52.238025 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:52.238027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:52.238028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:52.238236 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:52.238339 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.238536 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.238657 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:04:52.241710 5321 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:04:52.244920 5321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:52.260044 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.253686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46047 (local address 127.0.0.1:59418)
0807 05:04:52.254554 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.254624 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:52.254756 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:52.255584 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:52.255590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:52.255614 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:52.255765 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.255772 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.257304 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.257309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.258599 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.258607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.259455 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.259458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.259646 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.259680 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 05:04:52.259744 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 05:04:52.259886 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":148,"thread_start_us":75,"threads_started":1}
I20260807 05:04:52.260066 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.253962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59418 (local address 127.1.17.193:46047)
0807 05:04:52.255083 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.255091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.255115 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:04:52.255200 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.255205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.255409 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.255580 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:52.255875 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.257147 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.258717 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.259286 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.259589 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.259753 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.259804 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.259904 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":130,"threads_started":1}
I20260807 05:04:52.260406 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.254534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.1.17.194:34735)
0807 05:04:52.255137 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.255154 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.255172 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:04:52.255233 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.255237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.255409 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.255580 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:52.256122 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.256986 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.258389 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.259236 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.259350 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.259504 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.259860 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.259968 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":164,"threads_started":1}
I20260807 05:04:52.260377 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.254045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33769 (local address 127.0.0.1:41186)
0807 05:04:52.255003 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.255017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:52.255106 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:52.255779 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:52.255782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:52.255808 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:52.256027 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.256033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.257425 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.257428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.258657 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.258668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.259962 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.259965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.260069 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.260100 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 05:04:52.260180 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:04:52.260260 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":198,"spinlock_wait_cycles":10624,"thread_start_us":111,"threads_started":1}
I20260807 05:04:52.260751 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.254541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41186 (local address 127.1.17.195:33769)
0807 05:04:52.255330 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.255339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.255356 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:04:52.255420 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.255424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.255620 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.255764 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:52.256154 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.257287 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.258803 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.259640 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.259746 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.259882 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.260484 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.260593 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":183,"threads_started":1}
I20260807 05:04:52.261718 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.254004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:34735 (local address 127.0.0.1:56352)
0807 05:04:52.254988 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.255002 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:52.255106 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:52.255779 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:52.255782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:52.255810 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:52.255988 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.255996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.257146 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.257152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.258249 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.258259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.259377 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.259381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.259496 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.259630 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 05:04:52.259740 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 05:04:52.260345 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":552,"spinlock_wait_cycles":10368,"thread_start_us":119,"threads_started":1}
I20260807 05:04:52.263688 5721 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8d24749b19425684871809def3a0e7 (DEFAULT_TABLE table=test-table [id=92ae45e4cb714d4f8c4ae346521762b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:52.264211 5584 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8d24749b19425684871809def3a0e7 (DEFAULT_TABLE table=test-table [id=92ae45e4cb714d4f8c4ae346521762b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:52.264897 5721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8d24749b19425684871809def3a0e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:52.265316 5584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8d24749b19425684871809def3a0e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:52.266157 5447 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8d24749b19425684871809def3a0e7 (DEFAULT_TABLE table=test-table [id=92ae45e4cb714d4f8c4ae346521762b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:52.267108 5447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8d24749b19425684871809def3a0e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:52.270944 5809 tablet_bootstrap.cc:492] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Bootstrap starting.
I20260807 05:04:52.272496 5810 tablet_bootstrap.cc:492] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Bootstrap starting.
I20260807 05:04:52.273393 5809 tablet_bootstrap.cc:654] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:52.274168 5811 tablet_bootstrap.cc:492] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991: Bootstrap starting.
I20260807 05:04:52.274791 5810 tablet_bootstrap.cc:654] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:52.275461 5809 log.cc:824] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:52.275974 5810 log.cc:824] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:52.276314 5811 tablet_bootstrap.cc:654] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:52.277163 5811 log.cc:824] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:52.277690 5809 tablet_bootstrap.cc:492] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: No bootstrap required, opened a new log
I20260807 05:04:52.277880 5810 tablet_bootstrap.cc:492] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: No bootstrap required, opened a new log
I20260807 05:04:52.277927 5809 ts_tablet_manager.cc:1397] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 05:04:52.278092 5810 ts_tablet_manager.cc:1397] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:04:52.279011 5811 tablet_bootstrap.cc:492] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991: No bootstrap required, opened a new log
I20260807 05:04:52.279183 5811 ts_tablet_manager.cc:1397] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:04:52.280947 5810 raft_consensus.cc:348] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.281392 5810 raft_consensus.cc:374] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:52.281451 5810 raft_consensus.cc:729] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d770563b176a48f9b16ffd8540f3f20e, State: Initialized, Role: FOLLOWER
I20260807 05:04:52.281898 5810 consensus_queue.cc:260] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.281947 5809 raft_consensus.cc:348] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.282212 5809 raft_consensus.cc:374] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:52.282271 5809 raft_consensus.cc:729] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e576c80fab0342dca6c5f97c1aa8fb1d, State: Initialized, Role: FOLLOWER
I20260807 05:04:52.282657 5654 heartbeater.cc:503] Master 127.1.17.254:41831 was elected leader, sending a full tablet report...
I20260807 05:04:52.282919 5810 ts_tablet_manager.cc:1428] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:04:52.283339 5811 raft_consensus.cc:348] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.283578 5811 raft_consensus.cc:374] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:52.283638 5811 raft_consensus.cc:729] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357bd6a6e8da43f381400aefc30de991, State: Initialized, Role: FOLLOWER
I20260807 05:04:52.283824 5809 consensus_queue.cc:260] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.284024 5811 consensus_queue.cc:260] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.284533 5791 heartbeater.cc:503] Master 127.1.17.254:41831 was elected leader, sending a full tablet report...
I20260807 05:04:52.284844 5809 ts_tablet_manager.cc:1428] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 05:04:52.285502 5811 ts_tablet_manager.cc:1428] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:04:52.285531 5514 heartbeater.cc:503] Master 127.1.17.254:41831 was elected leader, sending a full tablet report...
I20260807 05:04:52.399382 5815 raft_consensus.cc:482] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:52.399658 5815 raft_consensus.cc:504] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.400913 5815 leader_election.cc:288] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 357bd6a6e8da43f381400aefc30de991 (127.1.17.193:46047), e576c80fab0342dca6c5f97c1aa8fb1d (127.1.17.195:33769)
I20260807 05:04:52.405810 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.401239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46047 (local address 127.1.17.194:44777)
0807 05:04:52.401404 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.401424 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:52.401546 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:52.401960 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:52.401964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:52.401990 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:52.402142 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.402151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.403170 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.403172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.404419 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.404431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.405335 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.405337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.405460 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.405493 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 05:04:52.405573 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:04:52.405651 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":28}
I20260807 05:04:52.405819 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.401555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:44777 (local address 127.1.17.193:46047)
0807 05:04:52.401697 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.401700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.401712 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:04:52.401759 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.401761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.401834 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.401907 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:52.402274 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.403056 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.404575 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.405218 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.405341 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.405490 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.405590 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.405681 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 05:04:52.405862 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.401239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33769 (local address 127.1.17.194:46525)
0807 05:04:52.401968 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.401981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:52.402081 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:52.402492 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:52.402494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:52.402510 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:04:52.402603 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.402608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.403469 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.403472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.404621 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:52.404628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.405476 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.405480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:52.405597 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.405616 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:04:52.405682 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 05:04:52.405738 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":390,"thread_start_us":128,"threads_started":1}
I20260807 05:04:52.405907 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.401468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:46525 (local address 127.1.17.195:33769)
0807 05:04:52.401603 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.401609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.402079 (+ 470us) server_negotiation.cc:407] Connection header received
0807 05:04:52.402324 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.402327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.402385 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.402465 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:04:52.402726 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.403352 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.404727 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.405368 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.405476 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.405537 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.405713 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.405793 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:04:52.406503 5742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8d24749b19425684871809def3a0e7" candidate_uuid: "d770563b176a48f9b16ffd8540f3f20e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" is_pre_election: true
I20260807 05:04:52.406564 5467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8d24749b19425684871809def3a0e7" candidate_uuid: "d770563b176a48f9b16ffd8540f3f20e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "357bd6a6e8da43f381400aefc30de991" is_pre_election: true
I20260807 05:04:52.406827 5742 raft_consensus.cc:2454] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d770563b176a48f9b16ffd8540f3f20e in term 0.
I20260807 05:04:52.406847 5467 raft_consensus.cc:2454] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d770563b176a48f9b16ffd8540f3f20e in term 0.
I20260807 05:04:52.407299 5535 leader_election.cc:302] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d770563b176a48f9b16ffd8540f3f20e, e576c80fab0342dca6c5f97c1aa8fb1d; no voters:
I20260807 05:04:52.407606 5815 raft_consensus.cc:2781] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:52.407706 5815 raft_consensus.cc:482] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:52.407750 5815 raft_consensus.cc:3037] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:52.408954 5815 raft_consensus.cc:504] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.409359 5815 leader_election.cc:288] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [CANDIDATE]: Term 1 election: Requested vote from peers 357bd6a6e8da43f381400aefc30de991 (127.1.17.193:46047), e576c80fab0342dca6c5f97c1aa8fb1d (127.1.17.195:33769)
I20260807 05:04:52.409729 5467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8d24749b19425684871809def3a0e7" candidate_uuid: "d770563b176a48f9b16ffd8540f3f20e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "357bd6a6e8da43f381400aefc30de991"
I20260807 05:04:52.409777 5742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8d24749b19425684871809def3a0e7" candidate_uuid: "d770563b176a48f9b16ffd8540f3f20e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d"
I20260807 05:04:52.409878 5467 raft_consensus.cc:3037] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:52.409899 5742 raft_consensus.cc:3037] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:52.410949 5742 raft_consensus.cc:2454] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d770563b176a48f9b16ffd8540f3f20e in term 1.
I20260807 05:04:52.411012 5467 raft_consensus.cc:2454] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d770563b176a48f9b16ffd8540f3f20e in term 1.
I20260807 05:04:52.411326 5535 leader_election.cc:302] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d770563b176a48f9b16ffd8540f3f20e, e576c80fab0342dca6c5f97c1aa8fb1d; no voters:
I20260807 05:04:52.411540 5815 raft_consensus.cc:2781] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:52.411782 5815 raft_consensus.cc:686] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 1 LEADER]: Becoming Leader. State: Replica: d770563b176a48f9b16ffd8540f3f20e, State: Running, Role: LEADER
I20260807 05:04:52.412140 5815 consensus_queue.cc:237] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } }
I20260807 05:04:52.415247 5320 catalog_manager.cc:5495] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e reported cstate change: term changed from 0 to 1, leader changed from <none> to d770563b176a48f9b16ffd8540f3f20e (127.1.17.194). New cstate: current_term: 1 leader_uuid: "d770563b176a48f9b16ffd8540f3f20e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d770563b176a48f9b16ffd8540f3f20e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 } health_report { overall_health: UNKNOWN } } }
I20260807 05:04:52.447458 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.443629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.1.17.194:34735)
0807 05:04:52.443767 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.443771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.443787 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:04:52.443907 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.443910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.443979 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.444093 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:04:52.444316 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.445168 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.445979 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:52.446307 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:52.446397 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:52.446658 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:04:52.447116 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:04:52.447230 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.447263 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.447314 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 05:04:52.452940 5792 tablet.cc:2369] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:04:52.453521 5742 raft_consensus.cc:1261] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 1 FOLLOWER]: Refusing update from remote peer d770563b176a48f9b16ffd8540f3f20e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:52.453619 5467 raft_consensus.cc:1261] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 1 FOLLOWER]: Refusing update from remote peer d770563b176a48f9b16ffd8540f3f20e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:52.454238 5819 consensus_queue.cc:1035] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e576c80fab0342dca6c5f97c1aa8fb1d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:52.454423 5815 consensus_queue.cc:1035] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [LEADER]: Connected to new peer: Peer: permanent_uuid: "357bd6a6e8da43f381400aefc30de991" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:52.471174 5320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:52.471349 5320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:52.473798 5320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47487884f59849d8aa46e142046bf353: Sending DeleteTablet for 3 replicas of tablet 7c8d24749b19425684871809def3a0e7
I20260807 05:04:52.474656 5447 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c8d24749b19425684871809def3a0e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:52 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:59418
I20260807 05:04:52.474762 5721 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c8d24749b19425684871809def3a0e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:52 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:41186
I20260807 05:04:52.475162 5832 tablet_replica.cc:330] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991: stopping tablet replica
I20260807 05:04:52.475332 5584 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c8d24749b19425684871809def3a0e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:52 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:56352
I20260807 05:04:52.475677 5832 raft_consensus.cc:2229] T 7c8d24749b19425684871809def3a0e7 P 357bd6a6e8da43f381400aefc30de991 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:52.475795 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 5385
I20260807 05:04:52.477708 5833 tablet_replica.cc:330] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: stopping tablet replica
I20260807 05:04:52.478061 5833 raft_consensus.cc:2229] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:52.478389 5833 raft_consensus.cc:2258] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:52.478708 5834 tablet_replica.cc:330] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: stopping tablet replica
I20260807 05:04:52.479018 5834 raft_consensus.cc:2229] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:52.479373 5834 raft_consensus.cc:2258] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:52.480685 5834 ts_tablet_manager.cc:1905] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:52.481042 5833 ts_tablet_manager.cc:1905] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:52.483824 5833 ts_tablet_manager.cc:1918] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:04:52.483924 5833 log.cc:1196] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/7c8d24749b19425684871809def3a0e7
I20260807 05:04:52.484282 5833 ts_tablet_manager.cc:1939] T 7c8d24749b19425684871809def3a0e7 P e576c80fab0342dca6c5f97c1aa8fb1d: Deleting consensus metadata
I20260807 05:04:52.484412 5834 ts_tablet_manager.cc:1918] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:04:52.484586 5834 log.cc:1196] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/7c8d24749b19425684871809def3a0e7
I20260807 05:04:52.484989 5834 ts_tablet_manager.cc:1939] T 7c8d24749b19425684871809def3a0e7 P d770563b176a48f9b16ffd8540f3f20e: Deleting consensus metadata
I20260807 05:04:52.485536 5305 catalog_manager.cc:4841] TS e576c80fab0342dca6c5f97c1aa8fb1d (127.1.17.195:33769): tablet 7c8d24749b19425684871809def3a0e7 (table test-table [id=92ae45e4cb714d4f8c4ae346521762b8]) successfully deleted
I20260807 05:04:52.486295 5308 catalog_manager.cc:4841] TS d770563b176a48f9b16ffd8540f3f20e (127.1.17.194:34735): tablet 7c8d24749b19425684871809def3a0e7 (table test-table [id=92ae45e4cb714d4f8c4ae346521762b8]) successfully deleted
W20260807 05:04:52.486702 5307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:46047 (error 108)
I20260807 05:04:52.486959 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 5519
I20260807 05:04:52.489140 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.488381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46047 (local address 127.0.0.1:59428)
0807 05:04:52.488986 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.489054 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:46047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526}
W20260807 05:04:52.489370 5307 catalog_manager.cc:4584] TS 357bd6a6e8da43f381400aefc30de991 (127.1.17.193:46047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c8d24749b19425684871809def3a0e7: Network error: Client connection negotiation failed: client connection to 127.1.17.193:46047: connect: Connection refused (error 111)
I20260807 05:04:52.494231 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 5659
I20260807 05:04:52.500182 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 5289
2026-08-07T05:04:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2429 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T05:04:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:52Z Disabled control of system clock
I20260807 05:04:52.531828 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:45567
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:45567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 05:04:52.643433 5839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:52.643674 5839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:52.643713 5839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:52.647660 5839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:52.647727 5839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:52.647745 5839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:52.647764 5839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:52.647782 5839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:52.652422 5839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:45567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:45567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:52.653393 5839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:52.654703 5839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:52.663256 5845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:52.663265 5846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:52.663318 5839 server_base.cc:1034] running on GCE node
W20260807 05:04:52.663273 5848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:52.664026 5839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:52.665143 5839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:52.666307 5839 hybrid_clock.cc:648] HybridClock initialized: now 1786079092666277 us; error 41 us; skew 500 ppm
I20260807 05:04:52.668172 5839 webserver.cc:460] Webserver started at http://127.1.17.254:40725/ using document root <none> and password file <none>
I20260807 05:04:52.668735 5839 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:52.668843 5839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:52.669094 5839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:52.671080 5839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "2576c524a2254c0d989321ccd691eca8"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "c9a292c8f6f1f4319093b05e6fec1792"
server_key_iv: "d3ba0e8e3c7f32db62208c359004eb70"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.671624 5839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "2576c524a2254c0d989321ccd691eca8"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "c9a292c8f6f1f4319093b05e6fec1792"
server_key_iv: "d3ba0e8e3c7f32db62208c359004eb70"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.675203 5839 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260807 05:04:52.677589 5854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:52.678719 5839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:04:52.678871 5839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "2576c524a2254c0d989321ccd691eca8"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "c9a292c8f6f1f4319093b05e6fec1792"
server_key_iv: "d3ba0e8e3c7f32db62208c359004eb70"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.678984 5839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:52.683606 5839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:52.684175 5839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:52.684346 5839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:52.691568 5839 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:45567
I20260807 05:04:52.691619 5911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:45567 every 8 connection(s)
I20260807 05:04:52.692484 5839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:52.695657 5913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:52.697382 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 5839
I20260807 05:04:52.697525 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:52.697804 1095 external_mini_cluster.cc:1428] Setting key e388b8e2dcdbde1bbab99a7445c63db8
I20260807 05:04:52.702189 5913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8: Bootstrap starting.
I20260807 05:04:52.703300 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.699216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.1.17.254:45567)
0807 05:04:52.699826 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.699845 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.699879 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:04:52.700518 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.700535 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.700967 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.701300 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:52.702093 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:52.702098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:52.702121 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:52.702131 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:52.702184 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:52.702200 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:52.702345 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.702426 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.702653 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":144,"threads_started":1}
I20260807 05:04:52.704471 5913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:52.705308 5913 log.cc:824] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:52.707075 5913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8: No bootstrap required, opened a new log
I20260807 05:04:52.710671 5913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2576c524a2254c0d989321ccd691eca8" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45567 } }
I20260807 05:04:52.710875 5913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:52.710909 5913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2576c524a2254c0d989321ccd691eca8, State: Initialized, Role: FOLLOWER
I20260807 05:04:52.711262 5913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2576c524a2254c0d989321ccd691eca8" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45567 } }
I20260807 05:04:52.711369 5913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:52.711411 5913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:52.711508 5913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:52.712334 5913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2576c524a2254c0d989321ccd691eca8" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45567 } }
I20260807 05:04:52.712649 5913 leader_election.cc:302] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2576c524a2254c0d989321ccd691eca8; no voters:
I20260807 05:04:52.713037 5913 leader_election.cc:288] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:52.713110 5918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:52.713341 5918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [term 1 LEADER]: Becoming Leader. State: Replica: 2576c524a2254c0d989321ccd691eca8, State: Running, Role: LEADER
I20260807 05:04:52.713645 5918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2576c524a2254c0d989321ccd691eca8" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45567 } }
I20260807 05:04:52.714284 5913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:52.715415 5919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2576c524a2254c0d989321ccd691eca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2576c524a2254c0d989321ccd691eca8" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45567 } } }
I20260807 05:04:52.715571 5919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:52.715404 5921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2576c524a2254c0d989321ccd691eca8. Latest consensus state: current_term: 1 leader_uuid: "2576c524a2254c0d989321ccd691eca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2576c524a2254c0d989321ccd691eca8" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45567 } } }
I20260807 05:04:52.715950 5921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:52.716579 5928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:52.719622 5928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:52.725021 5928 catalog_manager.cc:1347] Generated new cluster ID: ba35b8c821da440d9b9b26e476bb829e
I20260807 05:04:52.725081 5928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:52.731273 5928 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:52.731575 5928 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:52.738163 5928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8: Generated new TSK 0
I20260807 05:04:52.738786 5928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:04:52.747539 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45567
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 05:04:52.869478 5939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:52.869729 5939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:52.869764 5939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:52.873553 5939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:52.873600 5939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:52.873677 5939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:52.878459 5939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:52.879505 5939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:52.880884 5939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:52.891320 5946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:52.891379 5945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:52.891379 5939 server_base.cc:1034] running on GCE node
W20260807 05:04:52.891579 5948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:52.892004 5939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:52.892563 5939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:52.893739 5939 hybrid_clock.cc:648] HybridClock initialized: now 1786079092893705 us; error 46 us; skew 500 ppm
I20260807 05:04:52.895529 5939 webserver.cc:460] Webserver started at http://127.1.17.193:37865/ using document root <none> and password file <none>
I20260807 05:04:52.896109 5939 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:52.896193 5939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:52.896386 5939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:52.898474 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "c18a3885a902422fbacc384bed4b5f35"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "2da3e3f867729aa9adba61dce9e6a9c7"
server_key_iv: "d13a16bd63931bc3f623e82d0d0f774a"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.899029 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "c18a3885a902422fbacc384bed4b5f35"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "2da3e3f867729aa9adba61dce9e6a9c7"
server_key_iv: "d13a16bd63931bc3f623e82d0d0f774a"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.902562 5939 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:04:52.904860 5954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:52.906126 5939 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:52.906252 5939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "c18a3885a902422fbacc384bed4b5f35"
format_stamp: "Formatted at 2026-08-07 05:04:52 on dist-test-slave-1x4g"
server_key: "2da3e3f867729aa9adba61dce9e6a9c7"
server_key_iv: "d13a16bd63931bc3f623e82d0d0f774a"
server_key_version: "encryptionkey@0"
I20260807 05:04:52.906356 5939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:52.910808 5939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:52.911386 5939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:52.911561 5939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:52.912079 5939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:52.912994 5939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:52.913058 5939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:52.913125 5939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:52.913166 5939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:52.922061 5939 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:40181
I20260807 05:04:52.922098 6072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:40181 every 8 connection(s)
I20260807 05:04:52.922935 5939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:52.926097 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 5939
I20260807 05:04:52.926229 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:52.926496 1095 external_mini_cluster.cc:1428] Setting key 0789c9d24d58b08387904bf6c3cc83ed
I20260807 05:04:52.928517 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45567
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 05:04:52.929466 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.924649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:34625 (local address 127.1.17.254:45567)
0807 05:04:52.924829 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:52.924833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:52.925877 (+ 1044us) server_negotiation.cc:407] Connection header received
0807 05:04:52.927576 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:52.927579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:52.927638 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:52.927723 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:52.928978 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:52.928981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:52.928984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:52.928997 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:52.929027 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:52.929035 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:52.929129 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:52.929288 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 05:04:52.929334 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:04:52.930342 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:52.925087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45567 (local address 127.1.17.193:34625)
0807 05:04:52.925722 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:52.925762 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:52.927317 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:52.927910 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:52.927919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:52.928003 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:52.928604 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:52.928606 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:52.928682 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:52.928821 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:52.929128 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:52.929140 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:52.929149 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 05:04:52.929332 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 05:04:52.929582 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":110,"threads_started":1}
I20260807 05:04:52.931665 6073 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45567
I20260807 05:04:52.931888 6073 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:52.932271 6073 heartbeater.cc:511] Master 127.1.17.254:45567 requested a full tablet report, sending...
I20260807 05:04:52.933957 5872 ts_manager.cc:194] Registered new tserver with Master: c18a3885a902422fbacc384bed4b5f35 (127.1.17.193:40181)
W20260807 05:04:53.039831 6076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:53.040092 6076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:53.040132 6076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:53.044134 6076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:53.044181 6076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:53.044270 6076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:53.048975 6076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:53.050012 6076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:53.051430 6076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:53.060719 6084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:53.060775 6076 server_base.cc:1034] running on GCE node
W20260807 05:04:53.060729 6082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:53.060729 6081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:53.061391 6076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:53.061995 6076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:53.063184 6076 hybrid_clock.cc:648] HybridClock initialized: now 1786079093063159 us; error 47 us; skew 500 ppm
I20260807 05:04:53.064929 6076 webserver.cc:460] Webserver started at http://127.1.17.194:42833/ using document root <none> and password file <none>
I20260807 05:04:53.065541 6076 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:53.065634 6076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:53.065857 6076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:53.067795 6076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "f276a5682a514d969cf4eda5d195c74e"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "83a7faa98e1cc5df54a0d28eebad12a2"
server_key_iv: "832f015770323ece6c9a74ef13ea3db4"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.068352 6076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "f276a5682a514d969cf4eda5d195c74e"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "83a7faa98e1cc5df54a0d28eebad12a2"
server_key_iv: "832f015770323ece6c9a74ef13ea3db4"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.071900 6076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260807 05:04:53.074280 6090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:53.075521 6076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:53.075644 6076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "f276a5682a514d969cf4eda5d195c74e"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "83a7faa98e1cc5df54a0d28eebad12a2"
server_key_iv: "832f015770323ece6c9a74ef13ea3db4"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.075752 6076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:53.080279 6076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:53.080911 6076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:53.081086 6076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:53.081626 6076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:53.082557 6076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:53.082625 6076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:53.082693 6076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:53.082746 6076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:53.092075 6076 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:44135
I20260807 05:04:53.092097 6207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:44135 every 8 connection(s)
I20260807 05:04:53.092970 6076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:53.095779 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6076
I20260807 05:04:53.095911 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:53.096175 1095 external_mini_cluster.cc:1428] Setting key a98dd083a436eff57e8af8a4c1873888
I20260807 05:04:53.098367 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45567
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 05:04:53.099586 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.094736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:40501 (local address 127.1.17.254:45567)
0807 05:04:53.094884 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.094888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.095998 (+ 1110us) server_negotiation.cc:407] Connection header received
0807 05:04:53.097707 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.097710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.097743 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.097858 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.099052 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.099059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.099065 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.099082 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.099109 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.099122 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.099221 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.099345 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.099406 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:04:53.100525 6210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.095109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45567 (local address 127.1.17.194:40501)
0807 05:04:53.095827 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.095888 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.097471 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.098061 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.098069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.098159 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:53.098783 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:53.098785 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:53.098826 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:53.098895 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:53.099212 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:53.099220 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:53.099224 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:04:53.099352 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 05:04:53.099558 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":126,"threads_started":1}
I20260807 05:04:53.101701 6208 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45567
I20260807 05:04:53.101945 6208 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:53.102409 6208 heartbeater.cc:511] Master 127.1.17.254:45567 requested a full tablet report, sending...
I20260807 05:04:53.103564 5872 ts_manager.cc:194] Registered new tserver with Master: f276a5682a514d969cf4eda5d195c74e (127.1.17.194:44135)
W20260807 05:04:53.209225 6211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:53.209486 6211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:53.209555 6211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:53.213550 6211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:53.213595 6211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:53.213681 6211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:53.218556 6211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:53.219565 6211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:53.221001 6211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:53.230257 6219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:53.230260 6216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:53.230288 6217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:53.230500 6211 server_base.cc:1034] running on GCE node
I20260807 05:04:53.230881 6211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:53.231510 6211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:53.232668 6211 hybrid_clock.cc:648] HybridClock initialized: now 1786079093232656 us; error 24 us; skew 500 ppm
I20260807 05:04:53.234447 6211 webserver.cc:460] Webserver started at http://127.1.17.195:37813/ using document root <none> and password file <none>
I20260807 05:04:53.235055 6211 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:53.235118 6211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:53.235373 6211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:53.237524 6211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "ad829d46ffe943a3b0d1cf2d941a8222"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "699f90c12323bdfadc944401549cf04d"
server_key_iv: "05c1f2f6f0ed24f629a6b1467079d158"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.238144 6211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "ad829d46ffe943a3b0d1cf2d941a8222"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "699f90c12323bdfadc944401549cf04d"
server_key_iv: "05c1f2f6f0ed24f629a6b1467079d158"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.241889 6211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:04:53.244261 6225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:53.245514 6211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:53.245649 6211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "ad829d46ffe943a3b0d1cf2d941a8222"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "699f90c12323bdfadc944401549cf04d"
server_key_iv: "05c1f2f6f0ed24f629a6b1467079d158"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.245770 6211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:53.250428 6211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:53.251031 6211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:53.251246 6211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:53.251878 6211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:53.252892 6211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:53.252976 6211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:53.253062 6211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:53.253118 6211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:53.262529 6211 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:36201
I20260807 05:04:53.262573 6339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:36201 every 8 connection(s)
I20260807 05:04:53.263387 6211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:53.266184 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6211
I20260807 05:04:53.266307 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:53.266578 1095 external_mini_cluster.cc:1428] Setting key 43b5baeb090997d0f6be6e2b7eb6da67
I20260807 05:04:53.269625 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.265177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:36101 (local address 127.1.17.254:45567)
0807 05:04:53.265330 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.265335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.266420 (+ 1085us) server_negotiation.cc:407] Connection header received
0807 05:04:53.267894 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.267897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.267941 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.268024 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.269257 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.269261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.269263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.269273 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.269291 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.269300 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.269367 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.269496 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.269530 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:04:53.270468 6342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.265582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45567 (local address 127.1.17.195:36101)
0807 05:04:53.266226 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.266299 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.267708 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.268220 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.268229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.268339 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:53.269002 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:53.269003 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:53.269042 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:53.269109 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:53.269378 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:53.269385 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:53.269389 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:04:53.269516 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 05:04:53.269747 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260807 05:04:53.271492 6340 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45567
I20260807 05:04:53.271713 6340 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:53.272094 6340 heartbeater.cc:511] Master 127.1.17.254:45567 requested a full tablet report, sending...
I20260807 05:04:53.273072 5872 ts_manager.cc:194] Registered new tserver with Master: ad829d46ffe943a3b0d1cf2d941a8222 (127.1.17.195:36201)
I20260807 05:04:53.280339 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:04:53.283972 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.282219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.1.17.254:45567)
0807 05:04:53.282358 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.282362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.282545 (+ 183us) server_negotiation.cc:407] Connection header received
0807 05:04:53.282631 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.282633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.282661 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.282736 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.283536 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.283540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.283544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.283558 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.283576 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.283591 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.283673 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.283783 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.283814 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:04:53.286757 5872 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:04:53.290258 5872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:04:53.301108 6352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.298362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:44135 (local address 127.0.0.1:58126)
0807 05:04:53.298706 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.298754 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.298855 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.299635 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.299641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.299716 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:53.300295 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:53.300299 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:53.300325 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:53.300397 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:53.300762 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:53.300803 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:53.300810 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 05:04:53.300903 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 05:04:53.300967 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260807 05:04:53.301821 6353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.298501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.1.17.194:44135)
0807 05:04:53.298847 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.298863 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.298891 (+ 28us) server_negotiation.cc:407] Connection header received
0807 05:04:53.298963 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.298972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.299178 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.299327 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.300532 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.300538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.300550 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.300563 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.300612 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.300643 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.301500 (+ 857us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.301556 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.301639 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":136,"threads_started":1}
I20260807 05:04:53.302706 6354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.299069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:40181 (local address 127.0.0.1:37490)
0807 05:04:53.299644 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.299656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.300280 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.301465 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.301469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.301482 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:53.302063 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:53.302065 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:53.302076 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:53.302106 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:53.302446 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:53.302449 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:53.302453 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:04:53.302531 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 05:04:53.302571 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"spinlock_wait_cycles":283520,"thread_start_us":82,"threads_started":1}
I20260807 05:04:53.302707 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.299380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:36201 (local address 127.0.0.1:37152)
0807 05:04:53.299826 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.299840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.300263 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.300733 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.300736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.300750 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:53.301305 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:53.301307 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:53.301318 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:53.301348 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:53.302511 (+ 1163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:53.302515 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:53.302518 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:04:53.302582 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 05:04:53.302607 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":99,"threads_started":1}
I20260807 05:04:53.303022 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.299770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.1.17.195:36201)
0807 05:04:53.300131 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.300146 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.300167 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:04:53.300408 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.300427 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.300606 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.300749 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.302249 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.302255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.302267 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.302280 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.302323 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.302351 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.302469 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.302768 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.302842 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":154,"threads_started":1}
I20260807 05:04:53.304701 6138 tablet_service.cc:1467] Processing CreateTablet for tablet e5a33ad19be04cd09d43181502bd510d (DEFAULT_TABLE table=test-table [id=323c8117b2d845bc92f9ab98b476b650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:53.305372 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.299252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37490 (local address 127.1.17.193:40181)
0807 05:04:53.300367 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.300378 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.300397 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:04:53.300473 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.300480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.300666 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.300822 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.302249 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.302255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.302267 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.302279 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.302319 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.302348 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.302451 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.302590 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.305149 (+ 2559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002,"thread_start_us":103,"threads_started":1}
I20260807 05:04:53.305876 6138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5a33ad19be04cd09d43181502bd510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:53.305531 6273 tablet_service.cc:1467] Processing CreateTablet for tablet e5a33ad19be04cd09d43181502bd510d (DEFAULT_TABLE table=test-table [id=323c8117b2d845bc92f9ab98b476b650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:53.306519 6273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5a33ad19be04cd09d43181502bd510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:53.308023 6004 tablet_service.cc:1467] Processing CreateTablet for tablet e5a33ad19be04cd09d43181502bd510d (DEFAULT_TABLE table=test-table [id=323c8117b2d845bc92f9ab98b476b650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:04:53.309033 6004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5a33ad19be04cd09d43181502bd510d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:53.312556 6358 tablet_bootstrap.cc:492] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Bootstrap starting.
I20260807 05:04:53.313912 6359 tablet_bootstrap.cc:492] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Bootstrap starting.
I20260807 05:04:53.315061 6358 tablet_bootstrap.cc:654] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:53.315898 6358 log.cc:824] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:53.316085 6359 tablet_bootstrap.cc:654] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:53.316345 6360 tablet_bootstrap.cc:492] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: Bootstrap starting.
I20260807 05:04:53.316908 6359 log.cc:824] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:53.318229 6358 tablet_bootstrap.cc:492] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: No bootstrap required, opened a new log
I20260807 05:04:53.318454 6360 tablet_bootstrap.cc:654] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:53.318444 6358 ts_tablet_manager.cc:1397] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:04:53.318904 6359 tablet_bootstrap.cc:492] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: No bootstrap required, opened a new log
I20260807 05:04:53.319093 6359 ts_tablet_manager.cc:1397] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 05:04:53.319164 6360 log.cc:824] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:53.321018 6360 tablet_bootstrap.cc:492] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: No bootstrap required, opened a new log
I20260807 05:04:53.321202 6360 ts_tablet_manager.cc:1397] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:04:53.323200 6358 raft_consensus.cc:348] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.323519 6358 raft_consensus.cc:374] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:53.323587 6358 raft_consensus.cc:729] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad829d46ffe943a3b0d1cf2d941a8222, State: Initialized, Role: FOLLOWER
I20260807 05:04:53.323523 6359 raft_consensus.cc:348] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.323760 6359 raft_consensus.cc:374] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:53.323822 6359 raft_consensus.cc:729] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f276a5682a514d969cf4eda5d195c74e, State: Initialized, Role: FOLLOWER
I20260807 05:04:53.324098 6358 consensus_queue.cc:260] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.324195 6359 consensus_queue.cc:260] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.324792 6340 heartbeater.cc:503] Master 127.1.17.254:45567 was elected leader, sending a full tablet report...
I20260807 05:04:53.324813 6360 raft_consensus.cc:348] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.325073 6360 raft_consensus.cc:374] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:53.325134 6360 raft_consensus.cc:729] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c18a3885a902422fbacc384bed4b5f35, State: Initialized, Role: FOLLOWER
I20260807 05:04:53.325150 6358 ts_tablet_manager.cc:1428] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 05:04:53.325539 6360 consensus_queue.cc:260] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.326157 6359 ts_tablet_manager.cc:1428] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260807 05:04:53.326165 6073 heartbeater.cc:503] Master 127.1.17.254:45567 was elected leader, sending a full tablet report...
I20260807 05:04:53.326468 6208 heartbeater.cc:503] Master 127.1.17.254:45567 was elected leader, sending a full tablet report...
I20260807 05:04:53.327293 6360 ts_tablet_manager.cc:1428] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 05:04:53.343847 6209 tablet.cc:2369] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:04:53.424072 6074 tablet.cc:2369] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:04:53.426764 6365 raft_consensus.cc:482] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:04:53.426966 6365 raft_consensus.cc:504] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.428050 6365 leader_election.cc:288] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c18a3885a902422fbacc384bed4b5f35 (127.1.17.193:40181), ad829d46ffe943a3b0d1cf2d941a8222 (127.1.17.195:36201)
I20260807 05:04:53.429934 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.428413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:39803 (local address 127.1.17.193:40181)
0807 05:04:53.428558 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.428562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.428578 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:04:53.428663 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.428669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.428712 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.428823 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.429563 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.429566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.429569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.429580 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.429603 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.429618 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.429708 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.429782 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.429820 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 05:04:53.429934 6210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.428296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:40181 (local address 127.1.17.194:39803)
0807 05:04:53.428463 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.428483 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.428571 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.428863 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.428867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.428886 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:53.429390 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:53.429392 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:53.429411 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:53.429438 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:53.429694 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:53.429697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:53.429700 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 05:04:53.429774 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:04:53.429812 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 05:04:53.430642 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.428746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:47693 (local address 127.1.17.195:36201)
0807 05:04:53.428895 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.428899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.429118 (+ 219us) server_negotiation.cc:407] Connection header received
0807 05:04:53.429246 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.429248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.429290 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.429372 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.430193 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.430197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.430199 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.430212 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.430233 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.430243 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.430332 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.430513 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.430550 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:04:53.430653 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.428615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:36201 (local address 127.1.17.194:47693)
0807 05:04:53.429002 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.429016 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.429132 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.429388 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.429391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.429404 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:53.429889 (+ 485us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:04:53.429891 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:53.429901 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:04:53.429937 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:53.430409 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:53.430411 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:53.430413 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 05:04:53.430487 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:04:53.430518 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":267,"threads_started":1}
I20260807 05:04:53.430595 6025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a33ad19be04cd09d43181502bd510d" candidate_uuid: "f276a5682a514d969cf4eda5d195c74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18a3885a902422fbacc384bed4b5f35" is_pre_election: true
I20260807 05:04:53.430909 6025 raft_consensus.cc:2454] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f276a5682a514d969cf4eda5d195c74e in term 0.
I20260807 05:04:53.431401 6093 leader_election.cc:302] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c18a3885a902422fbacc384bed4b5f35, f276a5682a514d969cf4eda5d195c74e; no voters:
I20260807 05:04:53.431493 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a33ad19be04cd09d43181502bd510d" candidate_uuid: "f276a5682a514d969cf4eda5d195c74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" is_pre_election: true
I20260807 05:04:53.431651 6365 raft_consensus.cc:2781] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:04:53.431741 6365 raft_consensus.cc:482] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:04:53.431790 6365 raft_consensus.cc:3037] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:53.431797 6294 raft_consensus.cc:2454] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f276a5682a514d969cf4eda5d195c74e in term 0.
I20260807 05:04:53.433004 6365 raft_consensus.cc:504] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.433333 6365 leader_election.cc:288] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [CANDIDATE]: Term 1 election: Requested vote from peers c18a3885a902422fbacc384bed4b5f35 (127.1.17.193:40181), ad829d46ffe943a3b0d1cf2d941a8222 (127.1.17.195:36201)
I20260807 05:04:53.433701 6025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a33ad19be04cd09d43181502bd510d" candidate_uuid: "f276a5682a514d969cf4eda5d195c74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18a3885a902422fbacc384bed4b5f35"
I20260807 05:04:53.433755 6294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a33ad19be04cd09d43181502bd510d" candidate_uuid: "f276a5682a514d969cf4eda5d195c74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad829d46ffe943a3b0d1cf2d941a8222"
I20260807 05:04:53.433869 6025 raft_consensus.cc:3037] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:53.433902 6294 raft_consensus.cc:3037] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:53.434895 6025 raft_consensus.cc:2454] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f276a5682a514d969cf4eda5d195c74e in term 1.
I20260807 05:04:53.435093 6294 raft_consensus.cc:2454] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f276a5682a514d969cf4eda5d195c74e in term 1.
I20260807 05:04:53.435218 6093 leader_election.cc:302] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c18a3885a902422fbacc384bed4b5f35, f276a5682a514d969cf4eda5d195c74e; no voters:
I20260807 05:04:53.435390 6365 raft_consensus.cc:2781] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:53.435633 6365 raft_consensus.cc:686] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 1 LEADER]: Becoming Leader. State: Replica: f276a5682a514d969cf4eda5d195c74e, State: Running, Role: LEADER
I20260807 05:04:53.435953 6365 consensus_queue.cc:237] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } }
I20260807 05:04:53.439081 5871 catalog_manager.cc:5495] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e reported cstate change: term changed from 0 to 1, leader changed from <none> to f276a5682a514d969cf4eda5d195c74e (127.1.17.194). New cstate: current_term: 1 leader_uuid: "f276a5682a514d969cf4eda5d195c74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f276a5682a514d969cf4eda5d195c74e" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 } health_report { overall_health: UNKNOWN } } }
I20260807 05:04:53.453577 6353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.452251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58130 (local address 127.1.17.194:44135)
0807 05:04:53.452362 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.452365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.452377 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:04:53.452509 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.452511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.452567 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.452634 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.453292 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.453295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.453297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.453307 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.453325 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.453332 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.453400 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.453458 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.453488 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:04:53.459244 6025 raft_consensus.cc:1261] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35 [term 1 FOLLOWER]: Refusing update from remote peer f276a5682a514d969cf4eda5d195c74e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:53.459255 6294 raft_consensus.cc:1261] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 1 FOLLOWER]: Refusing update from remote peer f276a5682a514d969cf4eda5d195c74e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:04:53.459918 6369 consensus_queue.cc:1035] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad829d46ffe943a3b0d1cf2d941a8222" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 36201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:53.460126 6365 consensus_queue.cc:1035] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c18a3885a902422fbacc384bed4b5f35" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 40181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:04:53.468928 6373 mvcc.cc:205] Tried to move back new op lower bound from 7315779966802116608 to 7315779966716252160. Current Snapshot: MvccSnapshot[applied={T|T < 7315779966802116608}]
I20260807 05:04:53.475507 5872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:53.475672 5872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:04:53.478026 5872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2576c524a2254c0d989321ccd691eca8: Sending DeleteTablet for 3 replicas of tablet e5a33ad19be04cd09d43181502bd510d
I20260807 05:04:53.478837 6273 tablet_service.cc:1514] Processing DeleteTablet for tablet e5a33ad19be04cd09d43181502bd510d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:53 UTC) from {username='slave'} at 127.0.0.1:37152
I20260807 05:04:53.478806 6138 tablet_service.cc:1514] Processing DeleteTablet for tablet e5a33ad19be04cd09d43181502bd510d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:53 UTC) from {username='slave'} at 127.0.0.1:58126
I20260807 05:04:53.479046 6004 tablet_service.cc:1514] Processing DeleteTablet for tablet e5a33ad19be04cd09d43181502bd510d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:04:53 UTC) from {username='slave'} at 127.0.0.1:37490
I20260807 05:04:53.479287 6382 tablet_replica.cc:330] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: stopping tablet replica
I20260807 05:04:53.479475 6383 tablet_replica.cc:330] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: stopping tablet replica
I20260807 05:04:53.479597 6382 raft_consensus.cc:2229] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:04:53.479810 6383 raft_consensus.cc:2229] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:04:53.480002 6384 tablet_replica.cc:330] T e5a33ad19be04cd09d43181502bd510d P c18a3885a902422fbacc384bed4b5f35: stopping tablet replica
I20260807 05:04:53.480038 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 5939
I20260807 05:04:53.480952 6383 raft_consensus.cc:2258] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:53.482522 6383 ts_tablet_manager.cc:1905] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:53.483134 6382 raft_consensus.cc:2258] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:04:53.484687 6382 ts_tablet_manager.cc:1905] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:04:53.485836 6383 ts_tablet_manager.cc:1918] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:04:53.486025 6383 log.cc:1196] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/e5a33ad19be04cd09d43181502bd510d
I20260807 05:04:53.486418 6383 ts_tablet_manager.cc:1939] T e5a33ad19be04cd09d43181502bd510d P ad829d46ffe943a3b0d1cf2d941a8222: Deleting consensus metadata
I20260807 05:04:53.487476 6382 ts_tablet_manager.cc:1918] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:04:53.487577 6382 log.cc:1196] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/e5a33ad19be04cd09d43181502bd510d
I20260807 05:04:53.487854 5857 catalog_manager.cc:4841] TS ad829d46ffe943a3b0d1cf2d941a8222 (127.1.17.195:36201): tablet e5a33ad19be04cd09d43181502bd510d (table test-table [id=323c8117b2d845bc92f9ab98b476b650]) successfully deleted
I20260807 05:04:53.487913 6382 ts_tablet_manager.cc:1939] T e5a33ad19be04cd09d43181502bd510d P f276a5682a514d969cf4eda5d195c74e: Deleting consensus metadata
I20260807 05:04:53.488888 5857 catalog_manager.cc:4841] TS f276a5682a514d969cf4eda5d195c74e (127.1.17.194:44135): tablet e5a33ad19be04cd09d43181502bd510d (table test-table [id=323c8117b2d845bc92f9ab98b476b650]) successfully deleted
W20260807 05:04:53.490504 5857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:40181 (error 108)
I20260807 05:04:53.490671 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6076
I20260807 05:04:53.492640 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.492350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:40181 (local address 127.0.0.1:37504)
0807 05:04:53.492489 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.492567 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 05:04:53.492852 5857 catalog_manager.cc:4584] TS c18a3885a902422fbacc384bed4b5f35 (127.1.17.193:40181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5a33ad19be04cd09d43181502bd510d: Network error: Client connection negotiation failed: client connection to 127.1.17.193:40181: connect: Connection refused (error 111)
I20260807 05:04:53.498471 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6211
I20260807 05:04:53.505311 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 5839
2026-08-07T05:04:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1004 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 05:04:53.527726 1095 mini_oidc.cc:152] Starting JWKS server
I20260807 05:04:53.529608 1095 webserver.cc:460] Webserver started at https://127.0.0.1:40081/ using document root <none> and password file <none>
I20260807 05:04:53.529815 1095 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 05:04:53.530099 1095 webserver.cc:460] Webserver started at http://127.0.0.1:36171/ using document root <none> and password file <none>
2026-08-07T05:04:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:53Z Disabled control of system clock
I20260807 05:04:53.540423 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39407
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:39407
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40081/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/testchainca.cert with env {}
W20260807 05:04:53.652158 6391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:53.652402 6391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:53.652441 6391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:53.656409 6391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:53.656459 6391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:53.656476 6391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:53.656491 6391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:53.656507 6391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:04:53.656522 6391 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:04:53.656535 6391 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40081/jwks.json
I20260807 05:04:53.661729 6391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:39407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40081/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:53.662642 6391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:53.663555 6391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:53.671757 6399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:53.671813 6391 server_base.cc:1034] running on GCE node
W20260807 05:04:53.671778 6396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:53.671762 6397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:53.672585 6391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:53.673643 6391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:53.674827 6391 hybrid_clock.cc:648] HybridClock initialized: now 1786079093674784 us; error 57 us; skew 500 ppm
I20260807 05:04:53.676611 6391 webserver.cc:460] Webserver started at http://127.1.17.254:43275/ using document root <none> and password file <none>
I20260807 05:04:53.677241 6391 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:53.677335 6391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:53.677561 6391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:53.679471 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "7e8b4c11096c4db881d8eafdf095539c"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "81ae1dc37aa1e5976373fcbb2be1edec"
server_key_iv: "c7d5361baa8af4eca8c534be1e918b59"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.680047 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "7e8b4c11096c4db881d8eafdf095539c"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "81ae1dc37aa1e5976373fcbb2be1edec"
server_key_iv: "c7d5361baa8af4eca8c534be1e918b59"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.683504 6391 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:04:53.686014 6405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:53.687132 6391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:53.687268 6391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "7e8b4c11096c4db881d8eafdf095539c"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "81ae1dc37aa1e5976373fcbb2be1edec"
server_key_iv: "c7d5361baa8af4eca8c534be1e918b59"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.687371 6391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:53.699414 6391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:53.699994 6391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:53.700177 6391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:53.707211 6391 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:39407
I20260807 05:04:53.707286 6458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:39407 every 8 connection(s)
I20260807 05:04:53.708112 6391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:53.711241 6459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:53.716346 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6391
I20260807 05:04:53.716472 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:53.716918 1095 external_mini_cluster.cc:1428] Setting key ab8437e9508bcfbd4959d69101cbc7c6
I20260807 05:04:53.716962 6459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c: Bootstrap starting.
I20260807 05:04:53.719537 6459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:53.720322 6459 log.cc:824] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:53.722221 6459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c: No bootstrap required, opened a new log
I20260807 05:04:53.725857 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.718228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.1.17.254:39407)
0807 05:04:53.718898 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.718911 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.718943 (+ 32us) server_negotiation.cc:407] Connection header received
0807 05:04:53.719541 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.719557 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.720000 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.720324 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:53.720769 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:53.721832 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:53.722795 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:53.723241 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:53.723423 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:53.724203 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:53.724208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:53.724220 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:53.724589 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:53.724643 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:53.724662 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:53.724787 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:53.724946 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 05:04:53.725188 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":177,"threads_started":1}
I20260807 05:04:53.725781 6459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8b4c11096c4db881d8eafdf095539c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39407 } }
I20260807 05:04:53.726054 6459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:53.726119 6459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e8b4c11096c4db881d8eafdf095539c, State: Initialized, Role: FOLLOWER
I20260807 05:04:53.726568 6459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8b4c11096c4db881d8eafdf095539c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39407 } }
I20260807 05:04:53.726715 6459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:53.726775 6459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:53.726866 6459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:53.727905 6459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8b4c11096c4db881d8eafdf095539c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39407 } }
I20260807 05:04:53.728240 6459 leader_election.cc:302] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e8b4c11096c4db881d8eafdf095539c; no voters:
I20260807 05:04:53.728611 6459 leader_election.cc:288] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:53.728650 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:04:53.728688 6464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:53.728927 6464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [term 1 LEADER]: Becoming Leader. State: Replica: 7e8b4c11096c4db881d8eafdf095539c, State: Running, Role: LEADER
I20260807 05:04:53.729225 6464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8b4c11096c4db881d8eafdf095539c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39407 } }
I20260807 05:04:53.729954 6459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:53.730997 6467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e8b4c11096c4db881d8eafdf095539c. Latest consensus state: current_term: 1 leader_uuid: "7e8b4c11096c4db881d8eafdf095539c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8b4c11096c4db881d8eafdf095539c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39407 } } }
I20260807 05:04:53.731124 6467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:53.731429 6466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e8b4c11096c4db881d8eafdf095539c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e8b4c11096c4db881d8eafdf095539c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39407 } } }
I20260807 05:04:53.731539 6466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:53.731565 6474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:53.734647 6474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:53.739949 6474 catalog_manager.cc:1347] Generated new cluster ID: 40a07a94d4534201ab8fc4eeec57b7c9
I20260807 05:04:53.740017 6474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:53.752470 6474 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:53.753633 6474 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:53.761257 6474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e8b4c11096c4db881d8eafdf095539c: Generated new TSK 0
I20260807 05:04:53.761899 6474 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 05:04:53.797843 6491 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 05:04:53.798012 6491 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 05:04:53.798520 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:53.778667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47590 (local address 127.1.17.254:39407)
0807 05:04:53.778794 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:53.778799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:53.779023 (+ 224us) server_negotiation.cc:407] Connection header received
0807 05:04:53.779206 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:53.779209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:53.779262 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:53.779366 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:04:53.779622 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:53.780448 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:53.781398 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:53.781740 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:53.781838 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:53.782161 (+ 323us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:04:53.798146 (+ 15985us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 05:04:53.798391 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 05:04:53.798584 6462 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 05:04:53.800006 6490 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:53.778593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39407 (local address 127.0.0.1:47590)
0807 05:04:53.778943 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:53.778956 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:53.779062 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:53.779398 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:53.779400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:53.779414 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:04:53.779514 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:53.779520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:53.780561 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:53.780564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:53.781285 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:53.781295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:53.781858 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:53.781861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:53.781941 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:53.781985 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:04:53.798358 (+ 16373us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 05:04:53.798625 (+ 267us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.17.254:39407: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":121,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.17.254:39407: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 05:04:53.800935 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6391
2026-08-07T05:04:53Z chronyd exiting
I20260807 05:04:53.840551 1095 test_util.cc:175] -----------------------------------------------
I20260807 05:04:53.840641 1095 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786079068077062-1095-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (314 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 05:04:53.841842 1095 mini_oidc.cc:152] Starting JWKS server
I20260807 05:04:53.842691 1095 webserver.cc:460] Webserver started at https://127.0.0.1:34019/ using document root <none> and password file <none>
I20260807 05:04:53.842774 1095 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 05:04:53.842989 1095 webserver.cc:460] Webserver started at http://127.0.0.1:43491/ using document root <none> and password file <none>
2026-08-07T05:04:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:53Z Disabled control of system clock
I20260807 05:04:53.853214 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:33541
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:41447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:33541
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34019/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/testchainca.cert with env {}
W20260807 05:04:53.965580 6497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:53.965840 6497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:53.965880 6497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:53.969799 6497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:53.969851 6497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:53.969866 6497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:53.969882 6497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:53.969895 6497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:04:53.969911 6497 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:04:53.969933 6497 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34019/jwks.json
I20260807 05:04:53.975350 6497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:41447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:33541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:33541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34019/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:53.976401 6497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:53.977455 6497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:53.986090 6504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:53.986070 6502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:53.986059 6506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:53.986418 6497 server_base.cc:1034] running on GCE node
I20260807 05:04:53.987080 6497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:53.988283 6497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:53.989480 6497 hybrid_clock.cc:648] HybridClock initialized: now 1786079093989445 us; error 44 us; skew 500 ppm
I20260807 05:04:53.991317 6497 webserver.cc:460] Webserver started at http://127.1.17.254:38819/ using document root <none> and password file <none>
I20260807 05:04:53.991887 6497 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:53.991950 6497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:53.992208 6497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:53.994269 6497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "33bb6e080ee44c0ea369b683f3ab67fc"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "94772c93ad41e520c2752080b5e3b845"
server_key_iv: "a3c6e3a6f05af1c40fdcd6e9beaa6af6"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.994867 6497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "33bb6e080ee44c0ea369b683f3ab67fc"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "94772c93ad41e520c2752080b5e3b845"
server_key_iv: "a3c6e3a6f05af1c40fdcd6e9beaa6af6"
server_key_version: "encryptionkey@0"
I20260807 05:04:53.998574 6497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:04:54.000984 6512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:54.002177 6497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:54.002306 6497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "33bb6e080ee44c0ea369b683f3ab67fc"
format_stamp: "Formatted at 2026-08-07 05:04:53 on dist-test-slave-1x4g"
server_key: "94772c93ad41e520c2752080b5e3b845"
server_key_iv: "a3c6e3a6f05af1c40fdcd6e9beaa6af6"
server_key_version: "encryptionkey@0"
I20260807 05:04:54.002413 6497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:54.031020 6497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:54.031699 6497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:54.031885 6497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:54.039094 6497 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:33541
I20260807 05:04:54.039119 6569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:33541 every 8 connection(s)
I20260807 05:04:54.040040 6497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:54.043031 6570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:54.048377 6570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc: Bootstrap starting.
I20260807 05:04:54.049247 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6497
I20260807 05:04:54.049356 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:54.049686 1095 external_mini_cluster.cc:1428] Setting key be5d06b9876bcf0ae85f0aaa9fc9926f
I20260807 05:04:54.051294 6570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:54.052048 6570 log.cc:824] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:54.053977 6570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc: No bootstrap required, opened a new log
I20260807 05:04:54.057402 6570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33541 } }
I20260807 05:04:54.057643 6570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:54.057711 6570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33bb6e080ee44c0ea369b683f3ab67fc, State: Initialized, Role: FOLLOWER
I20260807 05:04:54.058182 6570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33541 } }
I20260807 05:04:54.058334 6570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:54.058424 6570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:54.058542 6570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:54.058897 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:54.050767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.1.17.254:33541)
0807 05:04:54.051448 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:54.051474 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:54.051512 (+ 38us) server_negotiation.cc:407] Connection header received
0807 05:04:54.052140 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:54.052189 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:54.052717 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:54.053118 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:54.053555 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.054594 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.055505 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.056000 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.056166 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.057074 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:54.057079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:54.057090 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:54.057454 (+ 364us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:54.057510 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:54.057529 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:54.057641 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:54.057757 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 05:04:54.058062 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":138,"threads_started":1}
I20260807 05:04:54.059593 6570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33541 } }
I20260807 05:04:54.059937 6570 leader_election.cc:302] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33bb6e080ee44c0ea369b683f3ab67fc; no voters:
I20260807 05:04:54.060268 6570 leader_election.cc:288] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:54.060331 6576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:54.060575 6576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [term 1 LEADER]: Becoming Leader. State: Replica: 33bb6e080ee44c0ea369b683f3ab67fc, State: Running, Role: LEADER
I20260807 05:04:54.061161 6576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33541 } }
I20260807 05:04:54.061546 6570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:54.061685 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:04:54.062711 6578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33bb6e080ee44c0ea369b683f3ab67fc. Latest consensus state: current_term: 1 leader_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33541 } } }
I20260807 05:04:54.062839 6578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:54.063162 6577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33bb6e080ee44c0ea369b683f3ab67fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33541 } } }
I20260807 05:04:54.063273 6577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:54.063539 6585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:54.066610 6585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:54.071332 6585 catalog_manager.cc:1347] Generated new cluster ID: 4e232f55a36748da8ce02d28ec7f7ca4
I20260807 05:04:54.071393 6585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:54.092667 6585 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:54.093581 6585 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:54.099066 6585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33bb6e080ee44c0ea369b683f3ab67fc: Generated new TSK 0
I20260807 05:04:54.099705 6585 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 05:04:54.125800 6602 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 05:04:54.125873 6602 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 05:04:54.126413 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:54.111048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.1.17.254:33541)
0807 05:04:54.111204 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:54.111208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:54.111418 (+ 210us) server_negotiation.cc:407] Connection header received
0807 05:04:54.111555 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:54.111558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:54.111599 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:54.111676 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:04:54.111934 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.112426 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.113657 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.113866 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.113942 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.114265 (+ 323us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:04:54.126139 (+ 11874us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 05:04:54.126319 (+ 180us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41606: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 05:04:54.126488 6574 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41606: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 05:04:54.126626 6601 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:54.110989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:33541 (local address 127.0.0.1:41606)
0807 05:04:54.111320 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:54.111334 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:54.111451 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:54.111710 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:54.111713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:54.111720 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:04:54.111837 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:54.111843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.112549 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.112553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:54.113545 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:54.113556 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.113982 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.113986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:54.114083 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.114110 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:04:54.126293 (+ 12183us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 05:04:54.126460 (+ 167us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.17.254:33541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34019/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":142,"threads_started":1}
I20260807 05:04:54.127256 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6497
2026-08-07T05:04:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (326 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 05:04:54.168289 1095 mini_oidc.cc:152] Starting JWKS server
I20260807 05:04:54.169229 1095 webserver.cc:460] Webserver started at https://127.0.0.1:40615/ using document root <none> and password file <none>
I20260807 05:04:54.169343 1095 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 05:04:54.169618 1095 webserver.cc:460] Webserver started at http://127.0.0.1:39609/ using document root <none> and password file <none>
2026-08-07T05:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:54Z Disabled control of system clock
I20260807 05:04:54.179468 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:33963
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:33963
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40615/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 05:04:54.288213 6608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:54.288451 6608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:54.288491 6608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:54.292244 6608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:54.292295 6608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:54.292313 6608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:54.292332 6608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:54.292349 6608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:04:54.292367 6608 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:04:54.292383 6608 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40615/jwks.json
I20260807 05:04:54.297086 6608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:33963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:33963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40615/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:54.298003 6608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:54.299220 6608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:54.307587 6613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:54.307588 6616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:54.307636 6614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:54.307811 6608 server_base.cc:1034] running on GCE node
I20260807 05:04:54.308351 6608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:54.309444 6608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:54.310667 6608 hybrid_clock.cc:648] HybridClock initialized: now 1786079094310626 us; error 42 us; skew 500 ppm
I20260807 05:04:54.312547 6608 webserver.cc:460] Webserver started at http://127.1.17.254:37637/ using document root <none> and password file <none>
I20260807 05:04:54.313151 6608 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:54.313242 6608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:54.313472 6608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:54.315389 6608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "a5015f200ace462f9674c9809c06d13c"
format_stamp: "Formatted at 2026-08-07 05:04:54 on dist-test-slave-1x4g"
server_key: "8ff74d4d4812756387653b7d1aa80686"
server_key_iv: "894960c9baf0c4d6b9369df7e3a37eb1"
server_key_version: "encryptionkey@0"
I20260807 05:04:54.315958 6608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "a5015f200ace462f9674c9809c06d13c"
format_stamp: "Formatted at 2026-08-07 05:04:54 on dist-test-slave-1x4g"
server_key: "8ff74d4d4812756387653b7d1aa80686"
server_key_iv: "894960c9baf0c4d6b9369df7e3a37eb1"
server_key_version: "encryptionkey@0"
I20260807 05:04:54.319459 6608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 05:04:54.321911 6622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:54.323107 6608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:54.323248 6608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "a5015f200ace462f9674c9809c06d13c"
format_stamp: "Formatted at 2026-08-07 05:04:54 on dist-test-slave-1x4g"
server_key: "8ff74d4d4812756387653b7d1aa80686"
server_key_iv: "894960c9baf0c4d6b9369df7e3a37eb1"
server_key_version: "encryptionkey@0"
I20260807 05:04:54.323357 6608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:54.340734 6608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:54.341348 6608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:54.341547 6608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:54.348356 6608 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:33963
I20260807 05:04:54.348377 6677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:33963 every 8 connection(s)
I20260807 05:04:54.349247 6608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:54.352208 6678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:54.354816 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6608
I20260807 05:04:54.354957 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:54.355271 1095 external_mini_cluster.cc:1428] Setting key a5dd676762385f49ad4f115730822cac
I20260807 05:04:54.358212 6678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c: Bootstrap starting.
I20260807 05:04:54.360263 6678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:54.361141 6678 log.cc:824] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:54.363327 6678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c: No bootstrap required, opened a new log
I20260807 05:04:54.365072 6682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:54.356640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50138 (local address 127.1.17.254:33963)
0807 05:04:54.357323 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:54.357337 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:54.357388 (+ 51us) server_negotiation.cc:407] Connection header received
0807 05:04:54.358039 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:54.358056 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:54.358545 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:54.358894 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:54.359333 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.360277 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.361574 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.362181 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.362309 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.363209 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:54.363217 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:54.363254 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:54.363696 (+ 442us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:54.363776 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:54.363797 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:54.363878 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:54.364054 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 05:04:54.364332 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":138,"threads_started":1}
I20260807 05:04:54.367301 6678 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5015f200ace462f9674c9809c06d13c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33963 } }
I20260807 05:04:54.367561 6678 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:54.367638 6678 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5015f200ace462f9674c9809c06d13c, State: Initialized, Role: FOLLOWER
I20260807 05:04:54.368162 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:04:54.368089 6678 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5015f200ace462f9674c9809c06d13c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33963 } }
I20260807 05:04:54.368263 6678 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:54.368332 6678 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:54.368422 6678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:54.369498 6678 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5015f200ace462f9674c9809c06d13c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33963 } }
I20260807 05:04:54.369864 6678 leader_election.cc:302] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5015f200ace462f9674c9809c06d13c; no voters:
I20260807 05:04:54.370219 6678 leader_election.cc:288] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:54.370298 6684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:54.370623 6684 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [term 1 LEADER]: Becoming Leader. State: Replica: a5015f200ace462f9674c9809c06d13c, State: Running, Role: LEADER
I20260807 05:04:54.370992 6684 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5015f200ace462f9674c9809c06d13c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33963 } }
I20260807 05:04:54.371485 6678 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:54.372633 6685 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5015f200ace462f9674c9809c06d13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5015f200ace462f9674c9809c06d13c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33963 } } }
I20260807 05:04:54.372798 6685 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:54.373201 6695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:54.373178 6687 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5015f200ace462f9674c9809c06d13c. Latest consensus state: current_term: 1 leader_uuid: "a5015f200ace462f9674c9809c06d13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5015f200ace462f9674c9809c06d13c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 33963 } } }
I20260807 05:04:54.373871 6687 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:54.376382 6695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:54.381901 6695 catalog_manager.cc:1347] Generated new cluster ID: 094a85966edf4ec5b78107dfeb2d830a
I20260807 05:04:54.382004 6695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:54.390604 6695 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:54.391474 6695 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:54.403429 6695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5015f200ace462f9674c9809c06d13c: Generated new TSK 0
I20260807 05:04:54.404312 6695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:04:54.426565 6682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:54.421303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.1.17.254:33963)
0807 05:04:54.421465 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:54.421473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:54.421634 (+ 161us) server_negotiation.cc:407] Connection header received
0807 05:04:54.421798 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:54.421801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:54.421854 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:54.421962 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:04:54.422230 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.422794 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.424021 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.424256 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.424347 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.424677 (+ 330us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:04:54.426024 (+ 1347us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40615/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 05:04:54.426390 (+ 366us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40615/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 05:04:54.426656 6682 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40615/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 05:04:54.429282 6712 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:54.421216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:33963 (local address 127.0.0.1:50142)
0807 05:04:54.421538 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:54.421554 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:54.421668 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:54.421985 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:54.421988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:54.421996 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:04:54.422121 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:54.422127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.422912 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.422916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:54.423888 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:54.423898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.424382 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.424386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:54.424482 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.424510 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:04:54.426256 (+ 1746us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40615/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 05:04:54.429136 (+ 2880us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.17.254:33963: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40615/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":2684,"thread_start_us":128,"threads_started":1}
I20260807 05:04:54.429837 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6608
E20260807 05:04:54.439868 6713 webserver.cc:533] Webserver: sslize failed:
E20260807 05:04:54.439954 6713 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T05:04:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (303 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T05:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:54Z Disabled control of system clock
I20260807 05:04:54.481317 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34655
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:44815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:34655
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/testchainca.cert with env {}
W20260807 05:04:54.590902 6717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:54.591136 6717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:54.591173 6717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:54.594992 6717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:54.595046 6717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:54.595062 6717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:54.595078 6717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:54.595093 6717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 05:04:54.595108 6717 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 05:04:54.595122 6717 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 05:04:54.600147 6717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:34655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:54.601166 6717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:54.602118 6717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:54.610885 6726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:54.610939 6724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:54.610898 6722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:54.611163 6717 server_base.cc:1034] running on GCE node
I20260807 05:04:54.611735 6717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:54.612730 6717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:54.613926 6717 hybrid_clock.cc:648] HybridClock initialized: now 1786079094613899 us; error 39 us; skew 500 ppm
I20260807 05:04:54.615772 6717 webserver.cc:460] Webserver started at http://127.1.17.254:43241/ using document root <none> and password file <none>
I20260807 05:04:54.616397 6717 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:54.616485 6717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:54.616724 6717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:54.618700 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c"
format_stamp: "Formatted at 2026-08-07 05:04:54 on dist-test-slave-1x4g"
server_key: "17e998c4f137316e65af930f6e98bd72"
server_key_iv: "3f42812c44649b3c63e889cbd2cabb77"
server_key_version: "encryptionkey@0"
I20260807 05:04:54.619247 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c"
format_stamp: "Formatted at 2026-08-07 05:04:54 on dist-test-slave-1x4g"
server_key: "17e998c4f137316e65af930f6e98bd72"
server_key_iv: "3f42812c44649b3c63e889cbd2cabb77"
server_key_version: "encryptionkey@0"
I20260807 05:04:54.622820 6717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:04:54.625200 6732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:54.626469 6717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:54.626600 6717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c"
format_stamp: "Formatted at 2026-08-07 05:04:54 on dist-test-slave-1x4g"
server_key: "17e998c4f137316e65af930f6e98bd72"
server_key_iv: "3f42812c44649b3c63e889cbd2cabb77"
server_key_version: "encryptionkey@0"
I20260807 05:04:54.626704 6717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:54.640959 6717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:54.641551 6717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:54.641721 6717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:54.648564 6717 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:34655
I20260807 05:04:54.648568 6790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:34655 every 8 connection(s)
I20260807 05:04:54.649542 6717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:54.652554 6792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:54.656992 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6717
I20260807 05:04:54.657101 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:54.657382 1095 external_mini_cluster.cc:1428] Setting key 3dc3b2eedb1d1b444f85b92544b29758
I20260807 05:04:54.658428 6792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c: Bootstrap starting.
I20260807 05:04:54.660705 6792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:54.661629 6792 log.cc:824] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:54.663475 6792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c: No bootstrap required, opened a new log
I20260807 05:04:54.666412 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:54.658797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.1.17.254:34655)
0807 05:04:54.659413 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:54.659428 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:54.659466 (+ 38us) server_negotiation.cc:407] Connection header received
0807 05:04:54.660084 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:54.660100 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:54.660480 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:54.660762 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:54.661221 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.662244 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.663100 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.663619 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.663794 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.664667 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:54.664672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:54.664684 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:04:54.665140 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:54.665195 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:04:54.665215 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:54.665324 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:54.665472 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 05:04:54.665727 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":131,"threads_started":1}
I20260807 05:04:54.667383 6792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34655 } }
I20260807 05:04:54.667655 6792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:54.667716 6792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca0b040c03a4845a0fa0dd9b9d9c47c, State: Initialized, Role: FOLLOWER
I20260807 05:04:54.668151 6792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34655 } }
I20260807 05:04:54.668341 6792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:54.668421 6792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:54.668524 6792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:54.669288 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:04:54.669701 6792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34655 } }
I20260807 05:04:54.670078 6792 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca0b040c03a4845a0fa0dd9b9d9c47c; no voters:
I20260807 05:04:54.670471 6792 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:54.670509 6798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:54.670704 6798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [term 1 LEADER]: Becoming Leader. State: Replica: 1ca0b040c03a4845a0fa0dd9b9d9c47c, State: Running, Role: LEADER
I20260807 05:04:54.671013 6798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34655 } }
I20260807 05:04:54.671711 6792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:54.672238 6801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34655 } } }
I20260807 05:04:54.672380 6801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:54.672726 6800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca0b040c03a4845a0fa0dd9b9d9c47c. Latest consensus state: current_term: 1 leader_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca0b040c03a4845a0fa0dd9b9d9c47c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34655 } } }
I20260807 05:04:54.672842 6804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:54.672894 6800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:54.675964 6804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:54.681370 6804 catalog_manager.cc:1347] Generated new cluster ID: 928879ffbd2d418e92bb43f7a82a9a5c
I20260807 05:04:54.681435 6804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:54.692703 6804 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:54.693647 6804 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:54.703270 6804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca0b040c03a4845a0fa0dd9b9d9c47c: Generated new TSK 0
I20260807 05:04:54.703966 6804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:04:55.155687 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:54.716855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.1.17.254:34655)
0807 05:04:54.716987 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:54.716991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:54.717190 (+ 199us) server_negotiation.cc:407] Connection header received
0807 05:04:54.717343 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:54.717346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:54.717391 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:54.717470 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0807 05:04:54.717720 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.718228 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.719488 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.719704 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.719774 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.720127 (+ 353us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 05:04:55.155242 (+435115us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 05:04:55.155560 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41228: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 05:04:55.155804 6796 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41228: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 05:04:55.155930 6825 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:54.716749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34655 (local address 127.0.0.1:41228)
0807 05:04:54.717099 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:54.717114 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:54.717227 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:54.717501 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:54.717504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:54.717512 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 05:04:54.717627 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:54.717634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.718339 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.718343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:54.719362 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:54.719373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:54.719820 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:54.719824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:54.719935 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:54.719964 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 05:04:55.155517 (+435553us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 05:04:55.155718 (+ 201us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.17.254:34655: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
I20260807 05:04:55.156649 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6717
2026-08-07T05:04:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (707 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:55 dist-test-slave-1x4g krb5kdc[6838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:55 dist-test-slave-1x4g krb5kdc[6838](info): set up 2 sockets
Aug 07 05:04:55 dist-test-slave-1x4g krb5kdc[6838](info): commencing operation
krb5kdc: starting...
W20260807 05:04:56.466543 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:56 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:04:56.589869 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36093
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:36093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:56.701112 6852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:56.701349 6852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:56.701388 6852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:56.705323 6852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:56.705369 6852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:56.705389 6852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:56.705405 6852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:56.705456 6852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:56.710534 6852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:36093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:56.711488 6852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:56.712450 6852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:56.720968 6858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:56.720947 6857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:56.721047 6852 server_base.cc:1034] running on GCE node
W20260807 05:04:56.720960 6860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:56.721782 6852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:56.722817 6852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:56.724012 6852 hybrid_clock.cc:648] HybridClock initialized: now 1786079096723986 us; error 38 us; skew 500 ppm
Aug 07 05:04:56 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079096, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:56.731690 6852 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:04:56.732558 6852 webserver.cc:460] Webserver started at http://127.1.17.254:41509/ using document root <none> and password file <none>
I20260807 05:04:56.733206 6852 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:56.733302 6852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:56.733560 6852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:56.735623 6852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "8a8e3db23468419dbc9f91e19f03a9e6"
format_stamp: "Formatted at 2026-08-07 05:04:56 on dist-test-slave-1x4g"
server_key: "660d10dc0174b4e2fdc49afd0bea52c4"
server_key_iv: "a0cd19af7490d7e0481fa29f373562bd"
server_key_version: "encryptionkey@0"
I20260807 05:04:56.736204 6852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "8a8e3db23468419dbc9f91e19f03a9e6"
format_stamp: "Formatted at 2026-08-07 05:04:56 on dist-test-slave-1x4g"
server_key: "660d10dc0174b4e2fdc49afd0bea52c4"
server_key_iv: "a0cd19af7490d7e0481fa29f373562bd"
server_key_version: "encryptionkey@0"
I20260807 05:04:56.739976 6852 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:04:56.742269 6868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:56.743481 6852 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:56.743634 6852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "8a8e3db23468419dbc9f91e19f03a9e6"
format_stamp: "Formatted at 2026-08-07 05:04:56 on dist-test-slave-1x4g"
server_key: "660d10dc0174b4e2fdc49afd0bea52c4"
server_key_iv: "a0cd19af7490d7e0481fa29f373562bd"
server_key_version: "encryptionkey@0"
I20260807 05:04:56.743743 6852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:56.755017 6852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:56.757707 6852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:56.757900 6852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:56.764731 6852 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:36093
I20260807 05:04:56.764746 6923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:36093 every 8 connection(s)
I20260807 05:04:56.765704 6852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:56.768399 6924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:56.773767 6924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6: Bootstrap starting.
I20260807 05:04:56.775919 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6852
I20260807 05:04:56.776022 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:56.776245 6924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:56.776350 1095 external_mini_cluster.cc:1428] Setting key 4c273af62b5e9ec8d7eeb0d721c078ee
I20260807 05:04:56.777180 6924 log.cc:824] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:56.779047 6924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6: No bootstrap required, opened a new log
I20260807 05:04:56.782441 6924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36093 } }
I20260807 05:04:56.782670 6924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:56.782749 6924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8e3db23468419dbc9f91e19f03a9e6, State: Initialized, Role: FOLLOWER
I20260807 05:04:56.783226 6924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36093 } }
I20260807 05:04:56.783399 6924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:56.783493 6924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:56.783613 6924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:56.784448 6924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36093 } }
Aug 07 05:04:56 dist-test-slave-1x4g krb5kdc[6838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:56.784828 6924 leader_election.cc:302] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a8e3db23468419dbc9f91e19f03a9e6; no voters:
I20260807 05:04:56.785211 6924 leader_election.cc:288] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:56.785327 6930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:56.785552 6930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [term 1 LEADER]: Becoming Leader. State: Replica: 8a8e3db23468419dbc9f91e19f03a9e6, State: Running, Role: LEADER
I20260807 05:04:56.785916 6930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36093 } }
I20260807 05:04:56.786630 6924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:56.788579 6932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a8e3db23468419dbc9f91e19f03a9e6. Latest consensus state: current_term: 1 leader_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36093 } } }
I20260807 05:04:56.788582 6931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8e3db23468419dbc9f91e19f03a9e6" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36093 } } }
I20260807 05:04:56.788702 6932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:56.788735 6931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:56.789065 6946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:56.791057 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:56.777498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59508 (local address 127.1.17.254:36093)
0807 05:04:56.777955 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:56.777980 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:56.778028 (+ 48us) server_negotiation.cc:407] Connection header received
0807 05:04:56.778639 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:56.778653 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:56.778941 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:56.779255 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:56.780036 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:56.781058 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:56.781880 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:56.782346 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:56.782524 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:56.785298 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:56.785305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:56.785324 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:56.785356 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:56.787395 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:56.787915 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:56.787920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:56.787926 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:56.788077 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:56.788439 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:56.788444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:56.788446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:56.789009 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:56.789228 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:56.789980 (+ 752us) server_negotiation.cc:299] Negotiation successful
0807 05:04:56.790205 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260807 05:04:56.792311 6946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:56.797251 6946 catalog_manager.cc:1347] Generated new cluster ID: 471b2977b1aa4721bd26d40cafc2336f
I20260807 05:04:56.797312 6946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:56.808524 6946 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:56.809509 6946 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:56.815269 6946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a8e3db23468419dbc9f91e19f03a9e6: Generated new TSK 0
I20260807 05:04:56.815944 6946 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:04:56.851785 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36093
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:56.964047 6952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:56.964309 6952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:56.964376 6952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:56.968287 6952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:56.968369 6952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:56.968456 6952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:56.973726 6952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:56.974741 6952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:56.975776 6952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:56.985102 6960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:56.985112 6958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:56.985112 6957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:56.985430 6952 server_base.cc:1034] running on GCE node
I20260807 05:04:56.985869 6952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:56.986492 6952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:56.987665 6952 hybrid_clock.cc:648] HybridClock initialized: now 1786079096987628 us; error 49 us; skew 500 ppm
Aug 07 05:04:56 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079096, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:56.990609 6952 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:04:56.991494 6952 webserver.cc:460] Webserver started at http://127.1.17.193:37015/ using document root <none> and password file <none>
I20260807 05:04:56.992112 6952 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:56.992183 6952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:56.992412 6952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:56.994453 6952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "4595ded6a3dc43f0a3686c5b1615034e"
format_stamp: "Formatted at 2026-08-07 05:04:56 on dist-test-slave-1x4g"
server_key: "8726ef81707831783b8c47c7f79f32de"
server_key_iv: "f9feff2e3c370f8c16e1c4c1e40c1f89"
server_key_version: "encryptionkey@0"
I20260807 05:04:56.995038 6952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "4595ded6a3dc43f0a3686c5b1615034e"
format_stamp: "Formatted at 2026-08-07 05:04:56 on dist-test-slave-1x4g"
server_key: "8726ef81707831783b8c47c7f79f32de"
server_key_iv: "f9feff2e3c370f8c16e1c4c1e40c1f89"
server_key_version: "encryptionkey@0"
I20260807 05:04:56.998875 6952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 05:04:57.001192 6967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.002368 6952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:57.002486 6952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "4595ded6a3dc43f0a3686c5b1615034e"
format_stamp: "Formatted at 2026-08-07 05:04:56 on dist-test-slave-1x4g"
server_key: "8726ef81707831783b8c47c7f79f32de"
server_key_iv: "f9feff2e3c370f8c16e1c4c1e40c1f89"
server_key_version: "encryptionkey@0"
I20260807 05:04:57.002583 6952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:57.023850 6952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:57.027187 6952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:57.027408 6952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:57.028005 6952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:57.028877 6952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:57.028950 6952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.029016 6952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:57.029049 6952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.038080 6952 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:36453
I20260807 05:04:57.038120 7088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:36453 every 8 connection(s)
I20260807 05:04:57.038957 6952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079096, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:57.048380 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 6952
I20260807 05:04:57.048513 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:57.048825 1095 external_mini_cluster.cc:1428] Setting key ad0cc5ab5a521b5211a66dedddb518f4
I20260807 05:04:57.052826 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.040735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:36515 (local address 127.1.17.254:36093)
0807 05:04:57.040874 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.040877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.041719 (+ 842us) server_negotiation.cc:407] Connection header received
0807 05:04:57.043061 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.043064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.043131 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.043243 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.044402 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.045016 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.046151 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.046390 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.046499 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.049116 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:57.049119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:57.049121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:57.049144 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:57.050738 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.051247 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.051251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.051253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.051341 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.051712 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.051716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.051718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.051902 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:57.052014 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:57.052503 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 05:04:57.052629 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:04:57.053610 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.041119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36093 (local address 127.1.17.193:36515)
0807 05:04:57.041569 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:57.041610 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:57.042848 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:57.043429 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:57.043435 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:57.043801 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:57.044254 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:57.044267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.045147 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.045151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:57.046023 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:57.046032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.046541 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.046545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:57.046706 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.047264 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:57.047279 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:57.048932 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:57.050858 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:57.050863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:57.050876 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:57.051141 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:57.051442 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:57.051445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:57.051447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:57.051612 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:57.052014 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:57.052022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:57.052264 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 05:04:57.052455 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 05:04:57.052769 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":118,"threads_started":1}
I20260807 05:04:57.054834 7089 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36093
I20260807 05:04:57.055112 7089 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:57.055727 7089 heartbeater.cc:511] Master 127.1.17.254:36093 requested a full tablet report, sending...
I20260807 05:04:57.057582 6886 ts_manager.cc:194] Registered new tserver with Master: 4595ded6a3dc43f0a3686c5b1615034e (127.1.17.193:36453)
I20260807 05:04:57.059279 6886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:36515
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:04:57.079059 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36093
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:57.189615 7094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:57.189872 7094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:57.189940 7094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:57.193878 7094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:57.193966 7094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:57.194044 7094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:57.199088 7094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:57.200057 7094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:57.201113 7094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:57.210843 7099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:57.210839 7100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:57.210836 7102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:57.210999 7094 server_base.cc:1034] running on GCE node
I20260807 05:04:57.211532 7094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:57.212141 7094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:57.213328 7094 hybrid_clock.cc:648] HybridClock initialized: now 1786079097213312 us; error 25 us; skew 500 ppm
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.216056 7094 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:04:57.216980 7094 webserver.cc:460] Webserver started at http://127.1.17.194:33641/ using document root <none> and password file <none>
I20260807 05:04:57.217540 7094 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:57.217736 7094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:57.217994 7094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:57.219933 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "a856b62c91bc483c9b7a50d12f7d1dde"
format_stamp: "Formatted at 2026-08-07 05:04:57 on dist-test-slave-1x4g"
server_key: "3e277c120adf258bfba5aa1546061f4d"
server_key_iv: "938e7a2dccba7ffe0959252550df18d9"
server_key_version: "encryptionkey@0"
I20260807 05:04:57.220477 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "a856b62c91bc483c9b7a50d12f7d1dde"
format_stamp: "Formatted at 2026-08-07 05:04:57 on dist-test-slave-1x4g"
server_key: "3e277c120adf258bfba5aa1546061f4d"
server_key_iv: "938e7a2dccba7ffe0959252550df18d9"
server_key_version: "encryptionkey@0"
I20260807 05:04:57.224076 7094 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:04:57.226490 7109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.227640 7094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:57.227792 7094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "a856b62c91bc483c9b7a50d12f7d1dde"
format_stamp: "Formatted at 2026-08-07 05:04:57 on dist-test-slave-1x4g"
server_key: "3e277c120adf258bfba5aa1546061f4d"
server_key_iv: "938e7a2dccba7ffe0959252550df18d9"
server_key_version: "encryptionkey@0"
I20260807 05:04:57.227901 7094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:57.264474 7094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:57.267485 7094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:57.267737 7094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:57.268352 7094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:57.269187 7094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:57.269258 7094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.269333 7094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:57.269383 7094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.278179 7094 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:36935
I20260807 05:04:57.278216 7230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:36935 every 8 connection(s)
I20260807 05:04:57.279019 7094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:57.285362 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 7094
I20260807 05:04:57.285478 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:57.285739 1095 external_mini_cluster.cc:1428] Setting key 140d563820f50fa1d18f803f6c2c3567
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:57.294755 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.280821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:44573 (local address 127.1.17.254:36093)
0807 05:04:57.280958 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.280962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.281786 (+ 824us) server_negotiation.cc:407] Connection header received
0807 05:04:57.283161 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.283165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.283229 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.283337 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.284457 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.285084 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.286397 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.286753 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.286866 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.290649 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:57.290652 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:57.290654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:57.290677 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:57.292219 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.292992 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.292997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.292999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.293098 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.293501 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.293505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.293507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.293715 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:57.293833 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:57.294434 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 05:04:57.294567 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:04:57.295620 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.281208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36093 (local address 127.1.17.194:44573)
0807 05:04:57.281638 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:57.281680 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:57.282887 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:57.283531 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:57.283538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:57.283864 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:57.284316 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:57.284329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.285228 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.285231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:57.286287 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:57.286298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.286918 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.286922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:57.287156 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.287826 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:57.287843 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:57.289781 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:57.292412 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:57.292419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:57.292435 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:57.292872 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:57.293212 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:57.293215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:57.293217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:57.293386 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:57.293846 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:57.293856 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:57.294144 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 05:04:57.294381 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 05:04:57.294720 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:04:57.296981 7231 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36093
I20260807 05:04:57.297242 7231 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:57.298123 7231 heartbeater.cc:511] Master 127.1.17.254:36093 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
I20260807 05:04:57.299479 6886 ts_manager.cc:194] Registered new tserver with Master: a856b62c91bc483c9b7a50d12f7d1dde (127.1.17.194:36935)
I20260807 05:04:57.300074 6886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:44573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:04:57.315110 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36093
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:57.441895 7236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:57.442135 7236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:57.442200 7236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:57.446349 7236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:57.446405 7236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:57.446496 7236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:57.453773 7236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:57.455552 7236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:57.457023 7236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:57.468449 7242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:57.468459 7244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:57.468986 7241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:57.469579 7236 server_base.cc:1034] running on GCE node
I20260807 05:04:57.470041 7236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:57.470929 7236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:57.472532 7236 hybrid_clock.cc:648] HybridClock initialized: now 1786079097472497 us; error 47 us; skew 500 ppm
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.476686 7236 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:04:57.477596 7236 webserver.cc:460] Webserver started at http://127.1.17.195:43899/ using document root <none> and password file <none>
I20260807 05:04:57.478349 7236 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:57.478430 7236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:57.478698 7236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:57.481140 7236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "b125d8d177304eeda5edd2ece2ad1233"
format_stamp: "Formatted at 2026-08-07 05:04:57 on dist-test-slave-1x4g"
server_key: "c656fe0eb0dacf6aadb30d38c97619ac"
server_key_iv: "4eb428391e80fa308d029044c542846e"
server_key_version: "encryptionkey@0"
I20260807 05:04:57.481722 7236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "b125d8d177304eeda5edd2ece2ad1233"
format_stamp: "Formatted at 2026-08-07 05:04:57 on dist-test-slave-1x4g"
server_key: "c656fe0eb0dacf6aadb30d38c97619ac"
server_key_iv: "4eb428391e80fa308d029044c542846e"
server_key_version: "encryptionkey@0"
I20260807 05:04:57.485618 7236 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:04:57.488125 7251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.489351 7236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:04:57.489503 7236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "b125d8d177304eeda5edd2ece2ad1233"
format_stamp: "Formatted at 2026-08-07 05:04:57 on dist-test-slave-1x4g"
server_key: "c656fe0eb0dacf6aadb30d38c97619ac"
server_key_iv: "4eb428391e80fa308d029044c542846e"
server_key_version: "encryptionkey@0"
I20260807 05:04:57.489631 7236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:57.504364 7236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:57.507632 7236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:57.507835 7236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:57.508417 7236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:57.509325 7236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:57.509371 7236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.509475 7236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:57.509514 7236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:57.518428 7236 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:44273
I20260807 05:04:57.518509 7370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:44273 every 8 connection(s)
I20260807 05:04:57.519310 7236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:04:57.522866 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 7236
I20260807 05:04:57.523015 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:57.523343 1095 external_mini_cluster.cc:1428] Setting key ec7cd4249af0e54087992712e35c3386
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:04:57.537766 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.521130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:32905 (local address 127.1.17.254:36093)
0807 05:04:57.521291 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.521296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.522164 (+ 868us) server_negotiation.cc:407] Connection header received
0807 05:04:57.523902 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.523904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.523951 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.524038 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.525734 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.526357 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.528130 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.528502 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.528565 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.532220 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:57.532226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:57.532230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:57.532257 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:57.534397 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.535049 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.535052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.535053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.535174 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.535544 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.535549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.535551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.535737 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:57.535947 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:57.537408 (+ 1461us) server_negotiation.cc:299] Negotiation successful
0807 05:04:57.537606 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:04:57.538424 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.521540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36093 (local address 127.1.17.195:32905)
0807 05:04:57.522009 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:57.522052 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:57.523669 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:57.524216 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:57.524228 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:57.524759 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:57.525544 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:57.525570 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.526493 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.526495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:57.527981 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:57.527996 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.528730 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.528732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:57.528991 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.529679 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:57.529706 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:57.532001 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:57.534533 (+ 2532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:57.534539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:57.534553 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:57.534850 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:57.535286 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:57.535289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:57.535291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:57.535444 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:57.535859 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:57.535866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:57.536485 (+ 619us) client_negotiation.cc:769] Sending connection context
0807 05:04:57.536785 (+ 300us) client_negotiation.cc:240] Negotiation successful
0807 05:04:57.537215 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260807 05:04:57.539695 7371 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36093
I20260807 05:04:57.540001 7371 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:57.540518 7371 heartbeater.cc:511] Master 127.1.17.254:36093 requested a full tablet report, sending...
I20260807 05:04:57.542019 6886 ts_manager.cc:194] Registered new tserver with Master: b125d8d177304eeda5edd2ece2ad1233 (127.1.17.195:44273)
I20260807 05:04:57.542634 6886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:32905
I20260807 05:04:57.550472 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/test-admin.keytab.
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.568735 1095 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 05:04:57.571555 7378 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:57.570136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.1.17.193:36453)
0807 05:04:57.570441 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.570455 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.570478 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:04:57.570530 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.570535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.570722 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.570872 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.571384 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59958: BlockingRecv error: recv got EOF from 127.0.0.1:59958 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":115,"threads_started":1}
W20260807 05:04:57.571630 1095 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5cc' of type FILE
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.572978 1095 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.573855 1095 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:57.582952 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.574671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59974 (local address 127.1.17.193:36453)
0807 05:04:57.574821 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.574824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.574888 (+ 64us) server_negotiation.cc:407] Connection header received
0807 05:04:57.575026 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.575028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.575068 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.575136 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.575614 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.576376 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.577257 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.577620 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.577701 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.579838 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:57.579847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:57.579859 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:57.579880 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:57.581353 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.581724 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.581729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.581731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.581806 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.582042 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.582044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.582045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.582334 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:57.582515 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:57.582645 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 05:04:57.582821 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 05:04:57.586671 1095 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5cc' of type FILE
W20260807 05:04:57.586683 7378 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:57.585558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.1.17.193:36453)
0807 05:04:57.585671 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.585675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.585771 (+ 96us) server_negotiation.cc:407] Connection header received
0807 05:04:57.585851 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.585853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.585887 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.585942 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.586598 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59986: BlockingRecv error: recv got EOF from 127.0.0.1:59986 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.587893 1095 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.588912 1095 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:57.596978 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.589743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.1.17.193:36453)
0807 05:04:57.589856 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.589861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.589974 (+ 113us) server_negotiation.cc:407] Connection header received
0807 05:04:57.590039 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.590041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.590085 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.590144 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.590555 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.590998 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.591667 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.591864 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.591933 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.594126 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:57.594145 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:57.594149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:57.594177 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:57.595345 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.595717 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.595720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.595721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.595788 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.596061 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.596065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.596067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.596296 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:57.596439 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:57.596681 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 05:04:57.596835 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 05:04:57.599458 7378 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:04:57.598420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.1.17.193:36453)
0807 05:04:57.598542 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.598546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.598657 (+ 111us) server_negotiation.cc:407] Connection header received
0807 05:04:57.598790 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.598793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.598827 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.598911 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.599379 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60006: BlockingRecv error: recv got EOF from 127.0.0.1:60006 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 05:04:57.599622 1095 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786079068077062-1095-0/krb5kdc/krb5cc' of type FILE
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.600629 1095 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:57.601544 1095 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[6838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:04:57.609246 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:57.602202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.1.17.193:36453)
0807 05:04:57.602323 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:57.602326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:57.602340 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:04:57.602435 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:57.602438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:57.602469 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:57.602528 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:57.602955 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.603389 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.604083 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:57.604267 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:57.604335 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:57.606429 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:57.606432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:57.606434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:57.606455 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:57.607837 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.608205 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.608208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.608209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.608287 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:57.608523 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:57.608526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:57.608527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:57.608670 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:57.608844 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:57.608981 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 05:04:57.609090 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:04:57.610430 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6952
I20260807 05:04:57.618718 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 7094
I20260807 05:04:57.624886 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 7236
I20260807 05:04:57.631896 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 6852
2026-08-07T05:04:57Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2218 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[7391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[7391](info): set up 2 sockets
Aug 07 05:04:57 dist-test-slave-1x4g krb5kdc[7391](info): commencing operation
krb5kdc: starting...
W20260807 05:04:58.682127 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:04:58 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:04:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:04:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab.
Entry for principal oryx/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab.
I20260807 05:04:58.807580 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35215
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab
--principal=oryx/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:35215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:58.923259 7406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:58.923527 7406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:58.923638 7406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:58.927620 7406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:58.927708 7406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:58.927745 7406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:58.927802 7406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:58.927858 7406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:58.933221 7406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:35215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab
--principal=oryx/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:58.934278 7406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:58.935305 7406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:58.943795 7416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:58.943768 7412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:58.943809 7413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:58.944021 7406 server_base.cc:1034] running on GCE node
I20260807 05:04:58.944628 7406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:58.945714 7406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:58.946916 7406 hybrid_clock.cc:648] HybridClock initialized: now 1786079098946902 us; error 27 us; skew 500 ppm
Aug 07 05:04:58 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079098, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:58.949839 7406 init.cc:377] Logged in from keytab as oryx/127.1.17.254@KRBTEST.COM (short username oryx)
I20260807 05:04:58.950742 7406 webserver.cc:460] Webserver started at http://127.1.17.254:41019/ using document root <none> and password file <none>
I20260807 05:04:58.951333 7406 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:58.951428 7406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:58.951685 7406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:58.953908 7406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "a3c4889c01ea4853ac4ec259fb340b20"
format_stamp: "Formatted at 2026-08-07 05:04:58 on dist-test-slave-1x4g"
server_key: "3e8958b5d457238683fa4017a445de44"
server_key_iv: "db5472b5cb73a681ef21bc22b1dcb792"
server_key_version: "encryptionkey@0"
I20260807 05:04:58.954473 7406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "a3c4889c01ea4853ac4ec259fb340b20"
format_stamp: "Formatted at 2026-08-07 05:04:58 on dist-test-slave-1x4g"
server_key: "3e8958b5d457238683fa4017a445de44"
server_key_iv: "db5472b5cb73a681ef21bc22b1dcb792"
server_key_version: "encryptionkey@0"
I20260807 05:04:58.958721 7406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:04:58.961228 7423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:58.962440 7406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:58.962598 7406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "a3c4889c01ea4853ac4ec259fb340b20"
format_stamp: "Formatted at 2026-08-07 05:04:58 on dist-test-slave-1x4g"
server_key: "3e8958b5d457238683fa4017a445de44"
server_key_iv: "db5472b5cb73a681ef21bc22b1dcb792"
server_key_version: "encryptionkey@0"
I20260807 05:04:58.962711 7406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:58.976390 7406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:58.979729 7406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:58.979935 7406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:58.987099 7406 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:35215
I20260807 05:04:58.987375 7481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:35215 every 8 connection(s)
I20260807 05:04:58.988085 7406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:04:58.990903 7482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:58.994026 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 7406
I20260807 05:04:58.994163 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:04:58.994438 1095 external_mini_cluster.cc:1428] Setting key 14a3729ffe7d09aca9d06a3d8e6ff46e
I20260807 05:04:58.996989 7482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: Bootstrap starting.
I20260807 05:04:58.999728 7482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:59.000595 7482 log.cc:824] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:59.002718 7482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: No bootstrap required, opened a new log
Aug 07 05:04:58 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:04:59.006170 7482 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } }
I20260807 05:04:59.006376 7482 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:59.006464 7482 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c4889c01ea4853ac4ec259fb340b20, State: Initialized, Role: FOLLOWER
I20260807 05:04:59.006911 7482 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } }
I20260807 05:04:59.007058 7482 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:04:59.007145 7482 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:04:59.007261 7482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:04:59.008404 7482 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } }
I20260807 05:04:59.008814 7482 leader_election.cc:302] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3c4889c01ea4853ac4ec259fb340b20; no voters:
I20260807 05:04:59.009145 7482 leader_election.cc:288] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:04:59.009245 7487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:04:59.009512 7487 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 1 LEADER]: Becoming Leader. State: Replica: a3c4889c01ea4853ac4ec259fb340b20, State: Running, Role: LEADER
I20260807 05:04:59.009862 7487 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } }
I20260807 05:04:59.010531 7482 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:04:59.011281 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:58.995865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.1.17.254:35215)
0807 05:04:58.996296 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:58.996314 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:58.996356 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:04:58.997248 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:58.997271 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:58.997678 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:58.998053 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:58.998702 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.000253 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.001459 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.002123 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.002277 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.004996 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:59.005002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:59.005015 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:59.005042 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:59.007547 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.008026 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.008030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.008035 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.008144 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.008458 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.008460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.008461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.008999 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:59.009223 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:59.009890 (+ 667us) server_negotiation.cc:299] Negotiation successful
0807 05:04:59.010197 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":108,"threads_started":1}
I20260807 05:04:59.011453 7488 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } }
I20260807 05:04:59.012063 7488 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:59.011678 7489 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c4889c01ea4853ac4ec259fb340b20. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } }
I20260807 05:04:59.012281 7489 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:04:59.012686 7496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:04:59.015806 7496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:04:59.021202 7496 catalog_manager.cc:1347] Generated new cluster ID: b59e5e2a51124417b8e7352683f68f51
I20260807 05:04:59.021267 7496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:04:59.033017 7496 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:04:59.034053 7496 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:04:59.044750 7496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: Generated new TSK 0
I20260807 05:04:59.045614 7496 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab.
Entry for principal oryx/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab.
I20260807 05:04:59.087941 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35215
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab
--principal=oryx/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:59.224514 7509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:59.224879 7509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:59.224943 7509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:59.228749 7509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:59.228817 7509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:59.228884 7509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:04:59.233835 7509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab
--principal=oryx/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:59.234874 7509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:59.235918 7509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:59.246061 7518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:59.246096 7515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:59.246104 7514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:59.246291 7509 server_base.cc:1034] running on GCE node
I20260807 05:04:59.246798 7509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:59.247502 7509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:59.248849 7509 hybrid_clock.cc:648] HybridClock initialized: now 1786079099248818 us; error 42 us; skew 500 ppm
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:59.251726 7509 init.cc:377] Logged in from keytab as oryx/127.1.17.193@KRBTEST.COM (short username oryx)
I20260807 05:04:59.252599 7509 webserver.cc:460] Webserver started at http://127.1.17.193:35769/ using document root <none> and password file <none>
I20260807 05:04:59.253235 7509 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:59.253330 7509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:59.253567 7509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:59.255633 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "da9149d2f3864246beb77be7325bd135"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "daf4ae482a13f3d7efe6353c84c18c96"
server_key_iv: "dcdc1283716eb12e80c0cc0d10a0cd65"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.256218 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "da9149d2f3864246beb77be7325bd135"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "daf4ae482a13f3d7efe6353c84c18c96"
server_key_iv: "dcdc1283716eb12e80c0cc0d10a0cd65"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.259951 7509 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:04:59.262318 7525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.263458 7509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:04:59.263602 7509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "da9149d2f3864246beb77be7325bd135"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "daf4ae482a13f3d7efe6353c84c18c96"
server_key_iv: "dcdc1283716eb12e80c0cc0d10a0cd65"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.263731 7509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:59.285434 7509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:59.288525 7509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:59.288715 7509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:59.289273 7509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:59.290055 7509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:59.290122 7509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.290189 7509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:59.290241 7509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.299152 7509 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:43031
I20260807 05:04:59.299180 7651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:43031 every 8 connection(s)
I20260807 05:04:59.300105 7509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:04:59.305051 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 7509
I20260807 05:04:59.305219 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:04:59.305543 1095 external_mini_cluster.cc:1428] Setting key f0de84620039d9fdc5cc1f16aeeba6bc
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:04:59.314189 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.301850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:56121 (local address 127.1.17.254:35215)
0807 05:04:59.301997 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:59.302001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:59.302869 (+ 868us) server_negotiation.cc:407] Connection header received
0807 05:04:59.304138 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:59.304141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:59.304188 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:59.304268 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:59.305535 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.306254 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.307637 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.308029 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.308165 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.310546 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:59.310549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:59.310552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:59.310584 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:59.312150 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.312663 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.312667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.312669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.312753 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.313137 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.313141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.313142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.313323 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:59.313430 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:59.313891 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 05:04:59.314022 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:04:59.314944 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.302292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:56121)
0807 05:04:59.302723 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:59.302764 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:59.303927 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:59.304421 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:59.304427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:59.304758 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:59.305379 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.305395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.306398 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.306401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.307513 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.307523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.308192 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.308194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.308332 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.308901 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:59.308917 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:59.310286 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:59.312269 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.312274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.312288 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.312557 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.312880 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.312882 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.312884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.313042 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.313428 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:59.313435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:59.313669 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 05:04:59.313849 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 05:04:59.314143 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for oryx/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:04:59.316304 7652 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35215
I20260807 05:04:59.316545 7652 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:59.317132 7652 heartbeater.cc:511] Master 127.1.17.254:35215 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.194@KRBTEST.COM" created.
I20260807 05:04:59.318696 7443 ts_manager.cc:194] Registered new tserver with Master: da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031)
I20260807 05:04:59.319792 7443 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:56121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab.
Entry for principal oryx/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab.
I20260807 05:04:59.334123 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35215
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab
--principal=oryx/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:59.444087 7658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:59.444327 7658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:59.444367 7658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:59.448187 7658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:59.448238 7658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:59.448293 7658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:04:59.453233 7658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab
--principal=oryx/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:59.454138 7658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:59.455087 7658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:59.464700 7664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:59.464681 7663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:59.464681 7666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:59.464670 7658 server_base.cc:1034] running on GCE node
I20260807 05:04:59.465353 7658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:59.465952 7658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:59.467109 7658 hybrid_clock.cc:648] HybridClock initialized: now 1786079099467095 us; error 26 us; skew 500 ppm
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:59.470022 7658 init.cc:377] Logged in from keytab as oryx/127.1.17.194@KRBTEST.COM (short username oryx)
I20260807 05:04:59.470851 7658 webserver.cc:460] Webserver started at http://127.1.17.194:42013/ using document root <none> and password file <none>
I20260807 05:04:59.471385 7658 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:59.471446 7658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:59.471665 7658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:59.473714 7658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "6696fca0c7dc4d709fd6621f51d20d0f"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "dabd29a1ee72f9fbded71ec3dfac2d78"
server_key_iv: "eb7f0c0d3cd9effcaee55fab2d979574"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.474268 7658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "6696fca0c7dc4d709fd6621f51d20d0f"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "dabd29a1ee72f9fbded71ec3dfac2d78"
server_key_iv: "eb7f0c0d3cd9effcaee55fab2d979574"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.477872 7658 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 05:04:59.480090 7674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.481423 7658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:04:59.481554 7658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "6696fca0c7dc4d709fd6621f51d20d0f"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "dabd29a1ee72f9fbded71ec3dfac2d78"
server_key_iv: "eb7f0c0d3cd9effcaee55fab2d979574"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.481657 7658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:59.494493 7658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:59.497309 7658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:59.497550 7658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:59.498117 7658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:59.498919 7658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:59.498986 7658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.499049 7658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:59.499099 7658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.508128 7658 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:39727
I20260807 05:04:59.508159 7794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:39727 every 8 connection(s)
I20260807 05:04:59.509128 7658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:04:59.510082 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 7658
I20260807 05:04:59.510196 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:04:59.510483 1095 external_mini_cluster.cc:1428] Setting key f097038bc458d3d1f4fd34e9f5860752
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.195@KRBTEST.COM" created.
I20260807 05:04:59.525655 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.511478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57403 (local address 127.1.17.254:35215)
0807 05:04:59.511599 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:59.511604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:59.512462 (+ 858us) server_negotiation.cc:407] Connection header received
0807 05:04:59.514224 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:59.514230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:59.514285 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:59.514365 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:59.515960 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.516467 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.518370 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.518598 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.518680 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.521524 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:59.521527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:59.521529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:59.521551 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:59.523302 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.523847 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.523850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.523852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.523931 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.524252 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.524255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.524256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.524430 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:59.524517 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:59.525273 (+ 756us) server_negotiation.cc:299] Negotiation successful
0807 05:04:59.525423 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 05:04:59.526424 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.511773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.194:57403)
0807 05:04:59.512289 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:59.512373 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:59.513945 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:59.514584 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:59.514595 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:59.515116 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:59.515810 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.515835 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.516597 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.516616 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.518238 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.518252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.518767 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.518773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.519032 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.519636 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:59.519657 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:59.521345 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:59.523433 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.523441 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.523456 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.523718 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.524023 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.524025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.524027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.524160 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.524540 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:59.524545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:59.524822 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 05:04:59.525076 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 05:04:59.525389 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":149,"threads_started":1}
I20260807 05:04:59.527722 7795 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35215
I20260807 05:04:59.527989 7795 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:59.528486 7795 heartbeater.cc:511] Master 127.1.17.254:35215 requested a full tablet report, sending...
I20260807 05:04:59.529639 7443 ts_manager.cc:194] Registered new tserver with Master: 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727)
I20260807 05:04:59.530229 7443 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:57403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab.
Entry for principal oryx/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab.
I20260807 05:04:59.541217 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35215
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab
--principal=oryx/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:59.652863 7800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:59.653175 7800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:59.653261 7800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:59.657317 7800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:59.657397 7800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:59.657485 7800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:04:59.662704 7800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab
--principal=oryx/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:04:59.663792 7800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:59.664950 7800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:59.674876 7809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:59.674885 7806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:59.674901 7805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:59.674924 7800 server_base.cc:1034] running on GCE node
I20260807 05:04:59.675630 7800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:59.676287 7800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:59.677492 7800 hybrid_clock.cc:648] HybridClock initialized: now 1786079099677457 us; error 52 us; skew 500 ppm
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:59.680382 7800 init.cc:377] Logged in from keytab as oryx/127.1.17.195@KRBTEST.COM (short username oryx)
I20260807 05:04:59.681317 7800 webserver.cc:460] Webserver started at http://127.1.17.195:41087/ using document root <none> and password file <none>
I20260807 05:04:59.681876 7800 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:59.681968 7800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:59.682207 7800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:59.684242 7800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "053d912d6fbf44cea636bdb2910c599a"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "30e41d699fa4e181f1e1464caf520b99"
server_key_iv: "a025dad24ccb6286f4c35ce512df8c54"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.684861 7800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "053d912d6fbf44cea636bdb2910c599a"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "30e41d699fa4e181f1e1464caf520b99"
server_key_iv: "a025dad24ccb6286f4c35ce512df8c54"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.688632 7800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:04:59.691084 7817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.692282 7800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:04:59.692423 7800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "053d912d6fbf44cea636bdb2910c599a"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "30e41d699fa4e181f1e1464caf520b99"
server_key_iv: "a025dad24ccb6286f4c35ce512df8c54"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.692531 7800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:59.703867 7800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:59.706682 7800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:59.706882 7800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:59.707460 7800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:04:59.708253 7800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:04:59.708329 7800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.708410 7800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:04:59.708454 7800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.717397 7800 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:38633
I20260807 05:04:59.717453 7936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:38633 every 8 connection(s)
I20260807 05:04:59.718237 7800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:04:59.727705 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 7800
I20260807 05:04:59.727833 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:04:59.728199 1095 external_mini_cluster.cc:1428] Setting key 1ace3743b58ecbabdbcb6c66857821b3
I20260807 05:04:59.731523 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.719877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:52097 (local address 127.1.17.254:35215)
0807 05:04:59.720023 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:59.720027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:59.720900 (+ 873us) server_negotiation.cc:407] Connection header received
0807 05:04:59.722196 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:59.722199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:59.722246 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:59.722318 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:59.723436 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.723968 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.725044 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.725274 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.725351 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.727892 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:59.727896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:59.727899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:59.727927 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:59.729527 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.730101 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.730104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.730105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.730179 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.730592 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.730595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.730596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.730784 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:59.730876 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:59.731298 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 05:04:59.731398 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:04:59.732237 7939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.720267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.195:52097)
0807 05:04:59.720699 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:59.720745 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:59.721995 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:59.722490 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:59.722495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:59.722817 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:59.723303 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.723317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.724075 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.724077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.724934 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.724943 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.725403 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.725405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.725541 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.726136 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:59.726153 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:59.727699 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:59.729620 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.729625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.729656 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.729985 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.730298 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.730302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.730305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.730493 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.730882 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:59.730887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:59.731094 (+ 207us) client_negotiation.cc:769] Sending connection context
0807 05:04:59.731242 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 05:04:59.731478 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":121,"threads_started":1}
I20260807 05:04:59.733373 7937 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35215
I20260807 05:04:59.733580 7937 heartbeater.cc:464] Registering TS with master...
I20260807 05:04:59.733999 7937 heartbeater.cc:511] Master 127.1.17.254:35215 requested a full tablet report, sending...
I20260807 05:04:59.734995 7443 ts_manager.cc:194] Registered new tserver with Master: 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633)
I20260807 05:04:59.735615 7443 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:52097
I20260807 05:04:59.742220 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.17.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.253.keytab.
Entry for principal oryx/127.1.17.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.253.keytab.
I20260807 05:04:59.771624 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:39489
--webserver_interface=127.1.17.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.253.keytab
--principal=oryx/127.1.17.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:35215,127.1.17.253:39489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:04:59.883332 7942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:04:59.883590 7942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:04:59.883628 7942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:04:59.887705 7942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:04:59.887759 7942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:04:59.887778 7942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:04:59.887794 7942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:04:59.887809 7942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:04:59.892763 7942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:35215,127.1.17.253:39489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.253.keytab
--principal=oryx/127.1.17.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:39489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.17.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
W20260807 05:04:59.892948 7942 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.17.254:35215,127.1.17.253:39489'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 05:04:59.893750 7942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:04:59.894694 7942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:04:59.903501 7948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:04:59.903487 7947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:59.903544 7942 server_base.cc:1034] running on GCE node
W20260807 05:04:59.903507 7950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:04:59.904311 7942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:04:59.905481 7942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:04:59.906673 7942 hybrid_clock.cc:648] HybridClock initialized: now 1786079099906655 us; error 33 us; skew 500 ppm
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:04:59.909546 7942 init.cc:377] Logged in from keytab as oryx/127.1.17.253@KRBTEST.COM (short username oryx)
I20260807 05:04:59.910631 7942 webserver.cc:460] Webserver started at http://127.1.17.253:43049/ using document root <none> and password file <none>
I20260807 05:04:59.911383 7942 fs_manager.cc:359] Metadata directory not provided
I20260807 05:04:59.911523 7942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:04:59.911823 7942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:04:59.914352 7942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/instance:
uuid: "64e07cde7c3f4ed295fcdac69c507145"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "a6b1202f89cd3ebe3ed8020cadb9bf63"
server_key_iv: "ace275525f18c825c1f702d9a8b870a3"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.914978 7942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal/instance:
uuid: "64e07cde7c3f4ed295fcdac69c507145"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "a6b1202f89cd3ebe3ed8020cadb9bf63"
server_key_iv: "ace275525f18c825c1f702d9a8b870a3"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.918655 7942 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:04:59.920904 7957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:04:59.922070 7942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:04:59.922188 7942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
uuid: "64e07cde7c3f4ed295fcdac69c507145"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "a6b1202f89cd3ebe3ed8020cadb9bf63"
server_key_iv: "ace275525f18c825c1f702d9a8b870a3"
server_key_version: "encryptionkey@0"
I20260807 05:04:59.922283 7942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:04:59.944439 7942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:04:59.947140 7942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:04:59.947326 7942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:04:59.954198 7942 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.253:39489
I20260807 05:04:59.954489 8012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.253:39489 every 8 connection(s)
I20260807 05:04:59.955242 7942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
I20260807 05:04:59.957742 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 7942
I20260807 05:04:59.958266 8014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:04:59.963229 8014 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.17.253@KRBTEST.COM
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.253@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:04:59.973320 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.959279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.1.17.253:39489)
0807 05:04:59.959706 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:59.959730 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:59.959771 (+ 41us) server_negotiation.cc:407] Connection header received
0807 05:04:59.960544 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:59.960567 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:59.961011 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:59.961353 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:59.961991 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.963423 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.964277 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.964931 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.965124 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.968187 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:59.968193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:59.968201 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:59.968226 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:59.970068 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.970570 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.970576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.970582 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.970732 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.971015 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.971019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.971022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.971440 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:59.971634 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:59.971945 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 05:04:59.972232 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":115,"threads_started":1}
I20260807 05:04:59.975008 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.965118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.1.17.254:35215)
0807 05:04:59.965270 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:59.965275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:59.965860 (+ 585us) server_negotiation.cc:407] Connection header received
0807 05:04:59.965952 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:59.965955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:59.966009 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:59.966106 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:59.966857 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.967455 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.968642 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.968941 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.969053 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.971441 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:59.971445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:59.971447 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:59.971475 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:59.973106 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.973637 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.973642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.973644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.973736 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.974002 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.974005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.974007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.974285 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:59.974389 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:59.974759 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 05:04:59.974875 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 05:04:59.975008 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.965064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:41646)
0807 05:04:59.965720 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:59.965776 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:59.965873 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:59.966152 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:59.966157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:59.966588 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:59.966734 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.966739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.967604 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.967609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.968521 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.968531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.969057 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.969060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.969178 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.969682 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:59.969694 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:59.971302 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:59.973207 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.973215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.973218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.973548 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.973828 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.973830 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.973832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.973904 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.974395 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:59.974400 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:59.974518 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 05:04:59.974677 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 05:04:59.974834 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":122,"threads_started":1}
I20260807 05:04:59.976183 8014 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:04:59 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079099, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.253@KRBTEST.COM for oryx/127.1.17.253@KRBTEST.COM
I20260807 05:04:59.983309 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.976626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.0.0.1:38188)
0807 05:04:59.976742 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 05:04:59.976757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:04:59.976843 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:04:59.977014 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:04:59.977016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:04:59.977136 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:04:59.977219 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.977223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.978225 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.978228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.978692 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:04:59.978699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.979083 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.979085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:04:59.979152 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.979493 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:04:59.979503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:04:59.980485 (+ 982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:04:59.982081 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.982084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.982085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.982258 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.982540 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:04:59.982543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:04:59.982545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:04:59.982636 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:04:59.982949 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:04:59.982952 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:04:59.983023 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 05:04:59.983102 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 05:04:59.983187 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 05:04:59.983372 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:04:59.976712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.1.17.253:39489)
0807 05:04:59.976835 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:04:59.976838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:04:59.976850 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:04:59.976909 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:04:59.976911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:04:59.976942 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:04:59.977007 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:04:59.977306 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.978127 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.978783 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:04:59.978995 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:04:59.979065 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:04:59.980616 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:04:59.980618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:04:59.980620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:04:59.980639 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:04:59.981969 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.982339 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.982341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.982342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.982404 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:04:59.982724 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:04:59.982726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:04:59.982727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:04:59.982867 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:04:59.982937 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:04:59.983155 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 05:04:59.983239 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:04:59.986233 8014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Bootstrap starting.
I20260807 05:04:59.988462 8014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Neither blocks nor log segments found. Creating new log.
I20260807 05:04:59.989245 8014 log.cc:824] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Log is configured to *not* fsync() on all Append() calls
I20260807 05:04:59.991029 8014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: No bootstrap required, opened a new log
I20260807 05:04:59.994096 8014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } }
I20260807 05:04:59.994323 8014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:04:59.994405 8014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e07cde7c3f4ed295fcdac69c507145, State: Initialized, Role: FOLLOWER
I20260807 05:04:59.994869 8014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } }
I20260807 05:04:59.995363 8020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } } }
I20260807 05:04:59.995507 8020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:04:59.996119 8014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 05:05:00.000746 8031 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:05:00.000893 8031 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:05:00.002874 7942 master_runner.cc:428] Detected that this master 64e07cde7c3f4ed295fcdac69c507145 is joining an existing cluster
I20260807 05:05:00.002939 7942 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.17.253:39489
I20260807 05:05:00.013731 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.005299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.1.17.254:35215)
0807 05:05:00.005450 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.005455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.005529 (+ 74us) server_negotiation.cc:407] Connection header received
0807 05:05:00.005626 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.005630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.005679 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.005769 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.006399 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.007212 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.008265 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.008499 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.008609 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.009875 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.009879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.009882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.009906 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.011889 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.012383 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.012385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.012386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.012521 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.012815 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.012818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.012820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.013063 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.013155 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.013467 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.013573 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:05:00.013747 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.004938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:41662)
0807 05:05:00.005466 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.005479 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.005558 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.005814 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.005817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.006060 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.006251 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.006259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.007320 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.007324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.008156 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.008163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.008621 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.008624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.008710 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.009262 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.009281 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.009742 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.012040 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.012044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.012046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.012288 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.012606 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.012609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.012611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.012700 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.013204 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.013207 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.013306 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.013414 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.013568 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":21,"thread_start_us":94,"threads_started":1}
I20260807 05:05:00.013979 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.005038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.1.17.253:39489)
0807 05:05:00.005136 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.005140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.005781 (+ 641us) server_negotiation.cc:407] Connection header received
0807 05:05:00.005936 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.005939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.005983 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.006063 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.006560 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.007019 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.008178 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.008519 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.008626 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.010165 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.010171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.010177 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.010207 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.011957 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.012805 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.012808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.012811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.012906 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.013159 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.013162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.013164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.013333 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.013394 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.013706 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.013832 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":36}
I20260807 05:05:00.014315 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.004995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.0.0.1:38190)
0807 05:05:00.005701 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.005710 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.005823 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.006097 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.006101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.006298 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.006452 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.006466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.007132 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.007135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.008047 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.008056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.008733 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.008736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.008866 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.009426 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.009442 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.009976 (+ 534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.012398 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.012400 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.012403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.012680 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.013005 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.013007 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.013009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.013080 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.013460 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.013462 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.013540 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.014049 (+ 509us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.014161 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":96,"threads_started":1}
I20260807 05:05:00.015653 8020 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:00.015800 8020 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } }
I20260807 05:05:00.016726 8020 leader_election.cc:288] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215)
I20260807 05:05:00.017256 7453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e07cde7c3f4ed295fcdac69c507145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c4889c01ea4853ac4ec259fb340b20" is_pre_election: true
I20260807 05:05:00.017980 7960 leader_election.cc:302] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 64e07cde7c3f4ed295fcdac69c507145; no voters: a3c4889c01ea4853ac4ec259fb340b20
I20260807 05:05:00.018383 8020 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079098, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.254@KRBTEST.COM for oryx/127.1.17.253@KRBTEST.COM
I20260807 05:05:00.029172 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.019127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.1.17.253:39489)
0807 05:05:00.019257 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.019262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.019649 (+ 387us) server_negotiation.cc:407] Connection header received
0807 05:05:00.019799 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.019801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.019841 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.019903 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.020629 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.021168 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.022415 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.022611 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.022681 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.025171 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.025174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.025176 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.025196 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.027083 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.027662 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.027666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.027668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.027751 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.028051 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.028055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.028057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.028298 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.028368 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.028886 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.029012 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 05:05:00.029721 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.019104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.0.0.1:38196)
0807 05:05:00.019509 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.019563 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.019694 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.019957 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.019963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.020383 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.020517 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.020525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.021268 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.021272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.022292 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.022303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.022707 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.022711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.022842 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.023320 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.023334 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.025022 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.027211 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.027221 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.027229 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.027545 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.027860 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.027863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.027866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.027950 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.028472 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.028480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.028609 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.029309 (+ 700us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.029543 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":119,"threads_started":1}
I20260807 05:05:00.031344 7443 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.17.253:39489: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145"
member_type: NON_VOTER
last_known_addr {
host: "127.1.17.253"
port: 39489
}
attrs {
promote: true
}
}
dest_uuid: "a3c4889c01ea4853ac4ec259fb340b20"
cas_config_opid_index: -1
I20260807 05:05:00.032063 7443 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } }
I20260807 05:05:00.034230 8041 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 64e07cde7c3f4ed295fcdac69c507145 (127.1.17.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } }
I20260807 05:05:00.034674 7985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:00.035893 8041 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.17.253:39489
I20260807 05:05:00.036087 8041 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } }
I20260807 05:05:00.036170 8041 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:00.036226 7985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Refusing update from remote peer a3c4889c01ea4853ac4ec259fb340b20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 05:05:00.036899 8041 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:00.036975 7942 master.cc:537] Master@127.1.17.253:39489 shutting down...
WARNING: no policy specified for oryx/127.1.17.252@KRBTEST.COM; defaulting to no policy
W20260807 05:05:00.038748 7425 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.252@KRBTEST.COM" created.
W20260807 05:05:00.040074 7425 consensus_peers.cc:487] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 -> Peer 64e07cde7c3f4ed295fcdac69c507145 (127.1.17.253:39489): Couldn't send request to peer 64e07cde7c3f4ed295fcdac69c507145. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 05:05:00.042070 7942 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:00.042344 7942 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:00.042457 7942 tablet_replica.cc:330] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: stopping tablet replica
I20260807 05:05:00.048265 7942 master.cc:552] Master@127.1.17.253:39489 shutdown complete.
I20260807 05:05:00.048321 7942 master_runner.cc:305] Clearing existing system tablet
I20260807 05:05:00.048574 7942 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.048633 7942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.050927 7942 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:05:00.052538 8047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.052865 7942 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.052997 7942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:00.053032 7942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
uuid: "64e07cde7c3f4ed295fcdac69c507145"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "a6b1202f89cd3ebe3ed8020cadb9bf63"
server_key_iv: "ace275525f18c825c1f702d9a8b870a3"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.055748 7942 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.252.keytab.
Entry for principal oryx/127.1.17.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.252.keytab.
I20260807 05:05:00.059248 7942 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 05:05:00.059348 7942 log.cc:1196] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 05:05:00.059423 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:36303
--webserver_interface=127.1.17.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.252.keytab
--principal=oryx/127.1.17.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:05:00.059613 7942 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Deleting consensus metadata
I20260807 05:05:00.059810 7942 master_runner.cc:315] Copying system tablet from 127.1.17.254:35215
I20260807 05:05:00.061069 7942 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: tablet copy: Beginning tablet copy session from remote peer at address 127.1.17.254:35215
I20260807 05:05:00.070317 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.061769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.1.17.254:35215)
0807 05:05:00.061917 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.061925 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.062029 (+ 104us) server_negotiation.cc:407] Connection header received
0807 05:05:00.062163 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.062168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.062232 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.062331 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.062834 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.063564 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.064752 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.065138 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.065214 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.066784 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.066787 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.066790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.066815 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.068516 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.069051 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.069055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.069057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.069136 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.069410 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.069425 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.069427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.069628 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.069695 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.070037 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.070150 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:05:00.070812 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.061627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:41670)
0807 05:05:00.061921 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.061936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.062063 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.062363 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.062366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.062585 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.062706 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.062713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.063680 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.063683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.064612 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.064623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.065377 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.065381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.065493 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.066144 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.066160 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.066631 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.068658 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.068661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.068663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.068940 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.069238 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.069241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.069243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.069321 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.069767 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.069770 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.069861 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.070476 (+ 615us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.070621 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":108,"threads_started":1}
I20260807 05:05:00.071475 7463 tablet_copy_service.cc:140] P a3c4889c01ea4853ac4ec259fb340b20: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 64e07cde7c3f4ed295fcdac69c507145 ({username='oryx', principal='oryx/127.1.17.253@KRBTEST.COM'} at 127.0.0.1:41670)
I20260807 05:05:00.071666 7463 tablet_copy_service.cc:161] P a3c4889c01ea4853ac4ec259fb340b20: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 64e07cde7c3f4ed295fcdac69c507145 at {username='oryx', principal='oryx/127.1.17.253@KRBTEST.COM'} at 127.0.0.1:41670: session id = 64e07cde7c3f4ed295fcdac69c507145-00000000000000000000000000000000
I20260807 05:05:00.073781 7463 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:05:00.075488 7942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 05:05:00.077854 7942 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: tablet copy: Starting download of 0 data blocks...
I20260807 05:05:00.078045 7942 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: tablet copy: Starting download of 1 WAL segments...
I20260807 05:05:00.079804 7942 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:05:00.081591 7463 tablet_copy_service.cc:342] P a3c4889c01ea4853ac4ec259fb340b20: Request end of tablet copy session 64e07cde7c3f4ed295fcdac69c507145-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.17.253@KRBTEST.COM'} at 127.0.0.1:41670
I20260807 05:05:00.081691 7463 tablet_copy_service.cc:433] P a3c4889c01ea4853ac4ec259fb340b20: ending tablet copy session 64e07cde7c3f4ed295fcdac69c507145-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 64e07cde7c3f4ed295fcdac69c507145
I20260807 05:05:00.090126 7942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:00.091801 8055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.092092 7942 server_base.cc:1034] running on GCE node
W20260807 05:05:00.092084 8060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.095798 8056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.096068 7942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:00.096370 7942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:00.097513 7942 hybrid_clock.cc:648] HybridClock initialized: now 1786079100097485 us; error 47 us; skew 500 ppm
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:00.099165 7942 init.cc:377] Logged in from keytab as oryx/127.1.17.253@KRBTEST.COM (short username oryx)
I20260807 05:05:00.099620 7942 webserver.cc:460] Webserver started at http://127.1.17.253:44089/ using document root <none> and password file <none>
I20260807 05:05:00.099802 7942 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.099852 7942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.101215 7942 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:00.102226 8068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.102516 7942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:00.102630 7942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
uuid: "64e07cde7c3f4ed295fcdac69c507145"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "a6b1202f89cd3ebe3ed8020cadb9bf63"
server_key_iv: "ace275525f18c825c1f702d9a8b870a3"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.102773 7942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.122126 7942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:00.124318 7942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:00.129331 7942 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.253:39489
I20260807 05:05:00.129364 8123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.253:39489 every 8 connection(s)
I20260807 05:05:00.130105 7942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
I20260807 05:05:00.132668 8124 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:05:00.133792 8124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Bootstrap starting.
I20260807 05:05:00.144232 8124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:05:00.144614 8124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Bootstrap complete.
I20260807 05:05:00.145251 8124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } }
I20260807 05:05:00.145418 8124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64e07cde7c3f4ed295fcdac69c507145, State: Initialized, Role: LEARNER
I20260807 05:05:00.145539 8124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } }
I20260807 05:05:00.145889 8127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } }
I20260807 05:05:00.145995 8124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:00.146087 8127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: LEARNER
I20260807 05:05:00.149353 8138 catalog_manager.cc:1259] Loaded cluster ID: b59e5e2a51124417b8e7352683f68f51
I20260807 05:05:00.149488 8138 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: loading cluster ID for follower catalog manager: success
I20260807 05:05:00.151739 8138 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: acquiring CA information for follower catalog manager: success
I20260807 05:05:00.153272 8138 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 05:05:00.191136 8053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:00.191406 8053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:00.191478 8053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:00.195345 8053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:00.195422 8053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:00.195456 8053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:00.195511 8053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:00.195550 8053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:00.200881 8053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.252.keytab
--principal=oryx/127.1.17.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:36303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.17.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:00.202020 8053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:00.203105 8053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:00.211436 8143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.211467 8144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.211567 8053 server_base.cc:1034] running on GCE node
W20260807 05:05:00.211508 8146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.212313 8053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:00.213465 8053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:00.214679 8053 hybrid_clock.cc:648] HybridClock initialized: now 1786079100214660 us; error 28 us; skew 500 ppm
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:00.217659 8053 init.cc:377] Logged in from keytab as oryx/127.1.17.252@KRBTEST.COM (short username oryx)
I20260807 05:05:00.218606 8053 webserver.cc:460] Webserver started at http://127.1.17.252:43425/ using document root <none> and password file <none>
I20260807 05:05:00.219162 8053 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.219256 8053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.219555 8053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:00.221629 8053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/instance:
uuid: "547729d48f9b447a982c25d1e8b645b7"
format_stamp: "Formatted at 2026-08-07 05:05:00 on dist-test-slave-1x4g"
server_key: "37e0a2d94d56178de8565215deea1519"
server_key_iv: "5113fd6cfa2dd45148976c9268a81baf"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.222239 8053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal/instance:
uuid: "547729d48f9b447a982c25d1e8b645b7"
format_stamp: "Formatted at 2026-08-07 05:05:00 on dist-test-slave-1x4g"
server_key: "37e0a2d94d56178de8565215deea1519"
server_key_iv: "5113fd6cfa2dd45148976c9268a81baf"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.226043 8053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:05:00.228477 8154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.229740 8053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:05:00.229903 8053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
uuid: "547729d48f9b447a982c25d1e8b645b7"
format_stamp: "Formatted at 2026-08-07 05:05:00 on dist-test-slave-1x4g"
server_key: "37e0a2d94d56178de8565215deea1519"
server_key_iv: "5113fd6cfa2dd45148976c9268a81baf"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.230019 8053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.244287 8053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:00.247282 8053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:00.247462 8053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:00.254523 8053 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.252:36303
I20260807 05:05:00.254806 8208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.252:36303 every 8 connection(s)
I20260807 05:05:00.255458 8053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
I20260807 05:05:00.258210 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 8053
I20260807 05:05:00.258347 8209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:00.263185 8209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.17.252@KRBTEST.COM
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.252@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:00.273384 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.259615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41224 (local address 127.1.17.252:36303)
0807 05:05:00.260001 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.260017 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.260051 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:05:00.260706 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.260721 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.261098 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.261437 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.262236 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.263224 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.264091 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.264523 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.264678 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.268028 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.268040 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.268049 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.268073 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.269899 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.270369 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.270375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.270381 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.270551 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.270837 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.270843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.270846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.271288 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.271560 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.272148 (+ 588us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.272426 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":111,"threads_started":1}
I20260807 05:05:00.274577 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.264906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41672 (local address 127.1.17.254:35215)
0807 05:05:00.265075 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.265081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.265324 (+ 243us) server_negotiation.cc:407] Connection header received
0807 05:05:00.265497 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.265501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.265555 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.265645 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.266556 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.267442 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.268525 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.268741 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.268897 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.271129 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.271132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.271134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.271155 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.272810 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.273271 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.273274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.273275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.273362 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.273673 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.273676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.273678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.273879 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.273978 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.274310 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.274424 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 05:05:00.274585 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.264702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:41672)
0807 05:05:00.265145 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.265227 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.265358 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.265724 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.265733 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.266245 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.266430 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.266442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.267577 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.267582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.268411 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.268420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.268899 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.268903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.269035 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.269445 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.269457 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.270987 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.272940 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.272948 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.272952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.273176 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.273480 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.273485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.273488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.273592 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.274016 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.274021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.274115 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.274233 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.274418 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":175,"threads_started":1}
I20260807 05:05:00.276147 8209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.252@KRBTEST.COM for oryx/127.1.17.253@KRBTEST.COM
I20260807 05:05:00.283797 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.276550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.0.0.1:38198)
0807 05:05:00.276689 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.276701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.276759 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.277248 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.277251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.277408 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.277495 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.277499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.278388 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.278397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.278923 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.278930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.279431 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.279434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.279510 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.279876 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.279885 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.281024 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.282565 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.282568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.282569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.282772 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.283037 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.283039 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.283040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.283118 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.283431 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.283433 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.283509 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.283598 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.283694 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 05:05:00.283851 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.276738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38198 (local address 127.1.17.253:39489)
0807 05:05:00.277035 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.277052 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.277064 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:05:00.277100 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.277102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.277139 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.277233 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.277620 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.278270 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.279040 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.279312 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.279410 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.281158 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.281161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.281163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.281183 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.282448 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.282869 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.282871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.282873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.282941 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.283195 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.283197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.283198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.283350 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.283421 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.283646 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.283737 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":139,"threads_started":1}
I20260807 05:05:00.284683 8209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.252@KRBTEST.COM for oryx/127.1.17.252@KRBTEST.COM
I20260807 05:05:00.292171 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.285206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.0.0.1:41228)
0807 05:05:00.285333 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.285361 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.285475 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.285828 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.285830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.285980 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.286041 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.286046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.287031 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.287034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.287479 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.287487 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.287885 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.287888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.287947 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.288392 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.288408 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.289563 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.290995 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.290997 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.290999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.291177 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.291406 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.291408 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.291409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.291482 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.291786 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.291788 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.291854 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.291943 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.292036 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 05:05:00.292196 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.285260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.1.17.252:36303)
0807 05:05:00.285360 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.285365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.285430 (+ 65us) server_negotiation.cc:407] Connection header received
0807 05:05:00.285565 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.285569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.285606 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.285732 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.286148 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.286918 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.287574 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.287761 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.287846 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.289688 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.289690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.289692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.289712 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.290896 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.291260 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.291262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.291263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.291330 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.291557 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.291559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.291560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.291714 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.291783 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.291983 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.292086 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":42}
I20260807 05:05:00.294835 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Bootstrap starting.
I20260807 05:05:00.297372 8209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:00.298161 8209 log.cc:824] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:00.300307 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: No bootstrap required, opened a new log
I20260807 05:05:00.304247 8209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } }
I20260807 05:05:00.304435 8209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:00.304471 8209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547729d48f9b447a982c25d1e8b645b7, State: Initialized, Role: FOLLOWER
I20260807 05:05:00.304888 8209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } }
I20260807 05:05:00.305346 8215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } } }
I20260807 05:05:00.305487 8215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:05:00.306025 8209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 05:05:00.310727 8227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:05:00.310806 8227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:05:00.314260 8053 master_runner.cc:428] Detected that this master 547729d48f9b447a982c25d1e8b645b7 is joining an existing cluster
I20260807 05:05:00.314324 8053 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.17.252:36303
I20260807 05:05:00.322063 7652 heartbeater.cc:503] Master 127.1.17.254:35215 was elected leader, sending a full tablet report...
I20260807 05:05:00.326467 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 7509
I20260807 05:05:00.337908 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:43031
--local_ip_for_outbound_sockets=127.1.17.193
--tserver_master_addrs=127.1.17.254:35215
--webserver_port=35769
--webserver_interface=127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab
--principal=oryx/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:05:00.350095 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.316248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.1.17.253:39489)
0807 05:05:00.316355 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.316362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.317078 (+ 716us) server_negotiation.cc:407] Connection header received
0807 05:05:00.317265 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.317270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.317313 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.317386 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.318079 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.318865 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.320430 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.320805 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.320878 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.346185 (+ 25307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.346190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.346193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.346226 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.347888 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.348605 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.348609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.348611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.348694 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.349037 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.349041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.349042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.349260 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.349341 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.349710 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.349861 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:05:00.350126 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.316192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:41676)
0807 05:05:00.316937 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.316949 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.317067 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.317408 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.317411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.317700 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.317942 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.317959 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.319004 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.319007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.320293 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.320303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.321047 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.321051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.321219 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.321765 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.321775 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.345479 (+ 23704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.347883 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.347888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.347891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.348234 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.348598 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.348603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.348607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.348709 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.349316 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.349320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.349506 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.349633 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.349819 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":270,"spinlock_wait_cycles":4352,"thread_start_us":100,"threads_started":1}
I20260807 05:05:00.350709 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.316230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.1.17.254:35215)
0807 05:05:00.316346 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.316351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.317028 (+ 677us) server_negotiation.cc:407] Connection header received
0807 05:05:00.317153 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.317157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.317205 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.317279 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.318077 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.318865 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.320430 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.320769 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.320878 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.345710 (+ 24832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.345715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.345721 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.345760 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.347707 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.348333 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.348340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.348344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.348438 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.348873 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.348876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.348878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.349156 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.349264 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.350367 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.350497 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:05:00.350828 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.316467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.0.0.1:38202)
0807 05:05:00.317000 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.317014 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.317107 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.317514 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.317516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.317710 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.317942 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.317950 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.319004 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.319007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.320293 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.320303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.321047 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.321051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.321336 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.345509 (+ 24173us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.345528 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.345974 (+ 446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.348020 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.348023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.348025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.348501 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.348844 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.348848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.348850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.348935 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.349423 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.349426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.349530 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.350473 (+ 943us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.350631 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":282,"spinlock_wait_cycles":49536,"thread_start_us":119,"threads_started":1}
I20260807 05:05:00.356122 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.316735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41242 (local address 127.1.17.252:36303)
0807 05:05:00.316858 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.316862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.317448 (+ 586us) server_negotiation.cc:407] Connection header received
0807 05:05:00.317592 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.317595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.317632 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.317678 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.318266 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.318998 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.320744 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.321087 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.321164 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.351166 (+ 30002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.351173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.351176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.351199 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.353919 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.354350 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.354353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.354355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.354422 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.354679 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.354682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.354684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.354861 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.354926 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.355821 (+ 895us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.355939 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":47,"spinlock_wait_cycles":4864}
I20260807 05:05:00.356397 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.316686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.0.0.1:41242)
0807 05:05:00.317365 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.317377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.317506 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.317824 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.317827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.318032 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.318147 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.318154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.319116 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.319118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.320619 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.320631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.321273 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.321276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.321363 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.322178 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.322192 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.348220 (+ 26028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.354035 (+ 5815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.354038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.354040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.354262 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.354524 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.354526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.354528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.354600 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.355529 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.355532 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.355638 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.355740 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.355876 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":91,"thread_start_us":107,"threads_started":1}
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079098, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.254@KRBTEST.COM for oryx/127.1.17.252@KRBTEST.COM
I20260807 05:05:00.365576 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.355149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.0.0.1:41254)
0807 05:05:00.355315 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.355328 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.355397 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.356410 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.356413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.356643 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.356756 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.356763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.357947 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.357951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.358846 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.358857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.359472 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.359476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.359591 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.360052 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.360067 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.361736 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.363905 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.363908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.363911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.364164 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.364464 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.364470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.364472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.364570 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.365011 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.365016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.365121 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.365239 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.365392 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 05:05:00.365904 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.355229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.1.17.252:36303)
0807 05:05:00.356113 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.356121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.356134 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:00.356205 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.356209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.356275 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.356889 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.356993 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.357827 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.358972 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.359346 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.359469 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.361920 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.361924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.361927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.361952 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.363786 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.364266 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.364270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.364271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.364356 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.364667 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.364670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.364672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.364907 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.365009 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.365309 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.365437 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":96,"threads_started":1}
I20260807 05:05:00.366962 7443 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.17.252:36303: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "547729d48f9b447a982c25d1e8b645b7"
member_type: NON_VOTER
last_known_addr {
host: "127.1.17.252"
port: 36303
}
attrs {
promote: true
}
}
dest_uuid: "a3c4889c01ea4853ac4ec259fb340b20"
cas_config_opid_index: 5
I20260807 05:05:00.367592 7443 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } }
I20260807 05:05:00.368597 8041 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.370067 8041 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.17.252:36303
I20260807 05:05:00.370240 8041 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.370340 8041 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:00.370941 8053 master.cc:537] Master@127.1.17.252:36303 shutting down...
W20260807 05:05:00.371866 7429 consensus_peers.cc:487] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 -> Peer 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252:36303): Couldn't send request to peer 547729d48f9b447a982c25d1e8b645b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 05:05:00.376420 8053 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:00.376721 8053 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:00.376847 8053 tablet_replica.cc:330] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: stopping tablet replica
I20260807 05:05:00.377612 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.370527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.1.17.253:39489)
0807 05:05:00.370697 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.370702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.372072 (+ 1370us) server_negotiation.cc:407] Connection header received
0807 05:05:00.372252 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.372255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.372314 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.372374 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:00.372755 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.373551 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.375975 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.376652 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.376760 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.377062 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.377395 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.377487 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:05:00.378114 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.368629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.0.0.1:38212)
0807 05:05:00.372005 (+ 3376us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.372019 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.372137 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.372533 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.372535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.372551 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:00.372629 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.372634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.373749 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.373752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.375857 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.375868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.377126 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.377130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.377227 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.377274 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.377917 (+ 643us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.377997 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3310}
I20260807 05:05:00.378882 8096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 LEARNER]: Refusing update from remote peer a3c4889c01ea4853ac4ec259fb340b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 05:05:00.379668 8042 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 05:05:00.381915 8096 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.383474 8053 master.cc:552] Master@127.1.17.252:36303 shutdown complete.
I20260807 05:05:00.383558 8053 master_runner.cc:305] Clearing existing system tablet
I20260807 05:05:00.383832 8053 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.383915 8053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.384121 8127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c4889c01ea4853ac4ec259fb340b20. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.384251 8127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: LEARNER
I20260807 05:05:00.384351 8127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: NON_VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: true } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.384426 8127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: LEARNER
I20260807 05:05:00.385950 8042 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: attempting to promote NON_VOTER 64e07cde7c3f4ed295fcdac69c507145 to VOTER
I20260807 05:05:00.386400 8042 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } }
I20260807 05:05:00.387701 8096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 LEARNER]: Refusing update from remote peer a3c4889c01ea4853ac4ec259fb340b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 05:05:00.388177 8241 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 05:05:00.388404 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.387513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.0.0.1:41262)
0807 05:05:00.388194 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.388271 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609}
I20260807 05:05:00.391013 8041 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 64e07cde7c3f4ed295fcdac69c507145 (127.1.17.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.391723 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.391261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.0.0.1:41264)
0807 05:05:00.391594 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.391646 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":81,"threads_started":1}
W20260807 05:05:00.391863 7429 consensus_peers.cc:487] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 -> Peer 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252:36303): Couldn't send request to peer 547729d48f9b447a982c25d1e8b645b7. Status: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 05:05:00.392129 8053 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.001s
I20260807 05:05:00.392611 8041 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.392715 8041 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:00.393208 8042 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.393318 8042 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:00.393904 8096 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 64e07cde7c3f4ed295fcdac69c507145 (127.1.17.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.395493 8239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.395620 8239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:05:00.395864 8244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.396124 8053 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.396256 8053 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260807 05:05:00.396301 8053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
uuid: "547729d48f9b447a982c25d1e8b645b7"
format_stamp: "Formatted at 2026-08-07 05:05:00 on dist-test-slave-1x4g"
server_key: "37e0a2d94d56178de8565215deea1519"
server_key_iv: "5113fd6cfa2dd45148976c9268a81baf"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.398469 8053 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:00.401031 8053 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 05:05:00.401136 8053 log.cc:1196] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 05:05:00.401430 8053 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Deleting consensus metadata
I20260807 05:05:00.401614 8053 master_runner.cc:315] Copying system tablet from 127.1.17.254:35215
I20260807 05:05:00.402777 8053 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: tablet copy: Beginning tablet copy session from remote peer at address 127.1.17.254:35215
I20260807 05:05:00.412753 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.403370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.1.17.254:35215)
0807 05:05:00.403488 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.403492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.403818 (+ 326us) server_negotiation.cc:407] Connection header received
0807 05:05:00.403937 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.403940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.404011 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.404094 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.404726 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.405587 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.407228 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.407588 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.407704 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.409116 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.409120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.409123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.409151 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.410815 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.411373 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.411377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.411379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.411479 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.411776 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.411780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.411810 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.412025 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.412100 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.412458 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.412579 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:05:00.413499 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.403316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:41686)
0807 05:05:00.403719 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.403734 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.403844 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.404226 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.404229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.404456 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.404600 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.404607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.405722 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.405726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.407073 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.407085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.407726 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.407729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.407824 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.408420 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.408435 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.408973 (+ 538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.410945 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.410949 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.410951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.411259 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.411592 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.411595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.411597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.411681 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.412174 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.412177 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.412271 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.413162 (+ 891us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.413312 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":111,"threads_started":1}
I20260807 05:05:00.414180 7463 tablet_copy_service.cc:140] P a3c4889c01ea4853ac4ec259fb340b20: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 547729d48f9b447a982c25d1e8b645b7 ({username='oryx', principal='oryx/127.1.17.252@KRBTEST.COM'} at 127.0.0.1:41686)
I20260807 05:05:00.414276 7463 tablet_copy_service.cc:161] P a3c4889c01ea4853ac4ec259fb340b20: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 547729d48f9b447a982c25d1e8b645b7 at {username='oryx', principal='oryx/127.1.17.252@KRBTEST.COM'} at 127.0.0.1:41686: session id = 547729d48f9b447a982c25d1e8b645b7-00000000000000000000000000000000
I20260807 05:05:00.415400 7463 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 05:05:00.416724 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 05:05:00.419351 8053 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: tablet copy: Starting download of 0 data blocks...
I20260807 05:05:00.419623 8053 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: tablet copy: Starting download of 1 WAL segments...
I20260807 05:05:00.421260 8053 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 05:05:00.423043 7463 tablet_copy_service.cc:342] P a3c4889c01ea4853ac4ec259fb340b20: Request end of tablet copy session 547729d48f9b447a982c25d1e8b645b7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.17.252@KRBTEST.COM'} at 127.0.0.1:41686
I20260807 05:05:00.423143 7463 tablet_copy_service.cc:433] P a3c4889c01ea4853ac4ec259fb340b20: ending tablet copy session 547729d48f9b447a982c25d1e8b645b7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 547729d48f9b447a982c25d1e8b645b7
I20260807 05:05:00.430634 8053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:00.435592 8254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.437215 8252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.439680 8053 server_base.cc:1034] running on GCE node
W20260807 05:05:00.440951 8251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.441222 8053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:00.441527 8053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:00.442678 8053 hybrid_clock.cc:648] HybridClock initialized: now 1786079100442651 us; error 41 us; skew 500 ppm
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:00.444433 8053 init.cc:377] Logged in from keytab as oryx/127.1.17.252@KRBTEST.COM (short username oryx)
I20260807 05:05:00.445019 8053 webserver.cc:460] Webserver started at http://127.1.17.252:41011/ using document root <none> and password file <none>
I20260807 05:05:00.445254 8053 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.445389 8053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.447192 8053 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 05:05:00.448405 8261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.448719 8053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:00.448858 8053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
uuid: "547729d48f9b447a982c25d1e8b645b7"
format_stamp: "Formatted at 2026-08-07 05:05:00 on dist-test-slave-1x4g"
server_key: "37e0a2d94d56178de8565215deea1519"
server_key_iv: "5113fd6cfa2dd45148976c9268a81baf"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.448969 8053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.465826 8053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:00.468266 8053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:00.473495 8053 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.252:36303
I20260807 05:05:00.473598 8315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.252:36303 every 8 connection(s)
I20260807 05:05:00.474084 8053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
I20260807 05:05:00.475881 8317 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:05:00.476974 8317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Bootstrap starting.
I20260807 05:05:00.488442 8317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:05:00.488761 8317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: Bootstrap complete.
I20260807 05:05:00.489390 8317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } }
I20260807 05:05:00.489585 8317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 547729d48f9b447a982c25d1e8b645b7, State: Initialized, Role: LEARNER
I20260807 05:05:00.489732 8317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } }
I20260807 05:05:00.490108 8323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.490234 8317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:00.490326 8323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: This master's current role is: LEARNER
W20260807 05:05:00.492372 8236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:00.492702 8236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:00.492799 8236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 05:05:00.494215 8336 catalog_manager.cc:1259] Loaded cluster ID: b59e5e2a51124417b8e7352683f68f51
I20260807 05:05:00.494277 8336 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: loading cluster ID for follower catalog manager: success
I20260807 05:05:00.496384 8336 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: acquiring CA information for follower catalog manager: success
I20260807 05:05:00.497841 8336 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 05:05:00.498507 8236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:00.498557 8236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:00.498661 8236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:05:00.504374 8236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab
--principal=oryx/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:43031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=35769
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:00.505522 8236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:00.506632 8236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:00.517495 8341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.517503 8339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.517508 8338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.517565 8236 server_base.cc:1034] running on GCE node
I20260807 05:05:00.518111 8236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:00.518668 8236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:00.519857 8236 hybrid_clock.cc:648] HybridClock initialized: now 1786079100519842 us; error 24 us; skew 500 ppm
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:00.522552 8236 init.cc:377] Logged in from keytab as oryx/127.1.17.193@KRBTEST.COM (short username oryx)
I20260807 05:05:00.523445 8236 webserver.cc:460] Webserver started at http://127.1.17.193:35769/ using document root <none> and password file <none>
I20260807 05:05:00.524024 8236 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.524143 8236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.527942 8236 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:00.530047 8348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.531219 8236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:05:00.531353 8236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "da9149d2f3864246beb77be7325bd135"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "daf4ae482a13f3d7efe6353c84c18c96"
server_key_iv: "dcdc1283716eb12e80c0cc0d10a0cd65"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.531880 8236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.532156 7795 heartbeater.cc:503] Master 127.1.17.254:35215 was elected leader, sending a full tablet report...
I20260807 05:05:00.543689 8236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:00.546653 8236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:00.546850 8236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:00.547561 8236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:00.548413 8236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:00.548489 8236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.548552 8236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:00.548606 8236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.558382 8236 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:43031
I20260807 05:05:00.558425 8474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:43031 every 8 connection(s)
I20260807 05:05:00.559342 8236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:05:00.567042 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 8236
I20260807 05:05:00.567240 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 7658
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.253@KRBTEST.COM
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.252@KRBTEST.COM
I20260807 05:05:00.578063 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.561427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:32827 (local address 127.1.17.254:35215)
0807 05:05:00.561555 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.561559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.562675 (+ 1116us) server_negotiation.cc:407] Connection header received
0807 05:05:00.563525 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.563529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.563588 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.563646 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.565137 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.565969 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.567708 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.568044 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.568130 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.572345 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.572348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.572351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.572377 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.575230 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.576680 (+ 1450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.576683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.576685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.576776 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.577141 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.577145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.577146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.577357 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.577437 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.577779 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.577896 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:05:00.578572 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.561194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:39151 (local address 127.1.17.252:36303)
0807 05:05:00.561704 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.561710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.562918 (+ 1208us) server_negotiation.cc:407] Connection header received
0807 05:05:00.563766 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.563769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.563811 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.563912 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.565121 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.565847 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.568285 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.568817 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.568958 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.572343 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.572347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.572350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.572377 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.574508 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.575160 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.575165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.575170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.575288 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.576006 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.576011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.576015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.576275 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.576379 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.578292 (+ 1913us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.578427 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":75,"threads_started":1}
I20260807 05:05:00.578737 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.561822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:35325 (local address 127.1.17.253:39489)
0807 05:05:00.561968 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.561973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.562372 (+ 399us) server_negotiation.cc:407] Connection header received
0807 05:05:00.563508 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.563511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.563571 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.563650 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.565138 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.565968 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.568168 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.568495 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.568570 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.572363 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.572370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.572377 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.572414 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.576165 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.577033 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.577039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.577044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.577141 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.577489 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.577495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.577500 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.577755 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.577845 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.578369 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.578502 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:05:00.578950 8481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.561600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.1.17.193:39151)
0807 05:05:00.562826 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.562839 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.563290 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.563932 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.563936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.564362 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.564975 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.564991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.566051 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.566053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.568103 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.568118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.569145 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.569152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.569290 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.569771 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.569795 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.572163 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.574661 (+ 2498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.574667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.574685 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.575025 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.575400 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.575405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.575408 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.575873 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.576446 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.576453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.576836 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.577075 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.577443 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1150,"mutex_wait_us":1011,"spinlock_wait_cycles":5376,"thread_start_us":116,"threads_started":1}
I20260807 05:05:00.579252 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.561600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:32827)
0807 05:05:00.562578 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.562590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.563284 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.563852 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.563860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.564370 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.564989 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.564999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.566164 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.566167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.567575 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.567589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.568331 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.568334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.568529 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.569421 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.569437 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.572204 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.576285 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.576289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.576291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.576532 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.576929 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.576932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.576934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.577030 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.577511 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.577514 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.577597 (+ 83us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.578233 (+ 636us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.578367 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":289,"spinlock_wait_cycles":41856,"thread_start_us":153,"threads_started":1}
I20260807 05:05:00.579757 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.561600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.1.17.193:35325)
0807 05:05:00.562227 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.562276 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.563284 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.563852 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.563860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.564374 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.564989 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.564999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.566120 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.566123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.568045 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.568054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.568724 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.568730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.568874 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.569413 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.569439 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.572131 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.576533 (+ 4402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.576539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.576544 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.576888 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.577262 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.577268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.577273 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.577380 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.577923 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.577941 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.578043 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.579183 (+ 1140us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.579592 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":845,"spinlock_wait_cycles":6144,"thread_start_us":121,"threads_started":1}
I20260807 05:05:00.580323 8475 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35215
I20260807 05:05:00.580547 8476 heartbeater.cc:347] Connected to a master server at 127.1.17.252:36303
I20260807 05:05:00.580588 8475 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:00.580621 8476 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:00.581045 8477 heartbeater.cc:347] Connected to a master server at 127.1.17.253:39489
I20260807 05:05:00.581128 8477 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:00.581200 8475 heartbeater.cc:511] Master 127.1.17.254:35215 requested a full tablet report, sending...
I20260807 05:05:00.581288 8477 heartbeater.cc:511] Master 127.1.17.253:39489 requested a full tablet report, sending...
I20260807 05:05:00.582013 8476 heartbeater.cc:511] Master 127.1.17.252:36303 requested a full tablet report, sending...
I20260807 05:05:00.582669 8085 ts_manager.cc:194] Registered new tserver with Master: da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031)
I20260807 05:05:00.583055 8270 ts_manager.cc:194] Registered new tserver with Master: da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031)
I20260807 05:05:00.583491 7443 ts_manager.cc:194] Re-registered known tserver with Master: da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031)
I20260807 05:05:00.584069 7443 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:32827
I20260807 05:05:00.585654 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:39727
--local_ip_for_outbound_sockets=127.1.17.194
--tserver_master_addrs=127.1.17.254:35215
--webserver_port=42013
--webserver_interface=127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab
--principal=oryx/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:00.700598 8484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:00.700918 8484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:00.700990 8484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:00.705343 8484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:00.705410 8484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:00.705502 8484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:05:00.711071 8484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab
--principal=oryx/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:39727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=42013
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:00.712316 8484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:00.713450 8484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:00.724014 8490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.724071 8489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.724035 8484 server_base.cc:1034] running on GCE node
W20260807 05:05:00.724037 8492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.724634 8484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:00.725283 8484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:00.726475 8484 hybrid_clock.cc:648] HybridClock initialized: now 1786079100726461 us; error 26 us; skew 500 ppm
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:00.729336 8484 init.cc:377] Logged in from keytab as oryx/127.1.17.194@KRBTEST.COM (short username oryx)
I20260807 05:05:00.730209 8484 webserver.cc:460] Webserver started at http://127.1.17.194:42013/ using document root <none> and password file <none>
I20260807 05:05:00.730785 8484 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.730913 8484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.734781 8484 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:05:00.737025 8501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.737805 7937 heartbeater.cc:503] Master 127.1.17.254:35215 was elected leader, sending a full tablet report...
I20260807 05:05:00.738534 8484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:05:00.738665 8484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "6696fca0c7dc4d709fd6621f51d20d0f"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "dabd29a1ee72f9fbded71ec3dfac2d78"
server_key_iv: "eb7f0c0d3cd9effcaee55fab2d979574"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.739388 8484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.752496 8484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:00.755926 8484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:00.756129 8484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:00.756734 8484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:00.757645 8484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:00.757694 8484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.757761 8484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:00.757791 8484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.767163 8484 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:39727
I20260807 05:05:00.767190 8620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:39727 every 8 connection(s)
I20260807 05:05:00.768143 8484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:05:00.772691 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 8484
I20260807 05:05:00.772888 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 7800
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.252@KRBTEST.COM
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.253@KRBTEST.COM
I20260807 05:05:00.784219 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:38633
--local_ip_for_outbound_sockets=127.1.17.195
--tserver_master_addrs=127.1.17.254:35215
--webserver_port=41087
--webserver_interface=127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab
--principal=oryx/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:05:00.787560 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.770084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:46401 (local address 127.1.17.252:36303)
0807 05:05:00.770221 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.770225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.771197 (+ 972us) server_negotiation.cc:407] Connection header received
0807 05:05:00.772279 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.772283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.772344 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.772450 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.774391 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.775286 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.777516 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.777945 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.778039 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.783112 (+ 5073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.783119 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.783125 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.783160 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.785307 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.785808 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.785813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.785816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.785908 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.786233 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.786237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.786241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.786476 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.786557 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.787269 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.787398 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:05:00.787560 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.770351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57339 (local address 127.1.17.253:39489)
0807 05:05:00.770500 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.770505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.771574 (+ 1069us) server_negotiation.cc:407] Connection header received
0807 05:05:00.772723 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.772727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.772807 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.772873 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.774392 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.775229 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.777510 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.777853 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.777927 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.783039 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.783043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.783046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.783074 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.785509 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.785989 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.785995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.785998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.786087 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.786405 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.786408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.786410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.786649 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.786718 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.787269 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.787393 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:05:00.787648 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.769990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:42161 (local address 127.1.17.254:35215)
0807 05:05:00.770118 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.770122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.771657 (+ 1535us) server_negotiation.cc:407] Connection header received
0807 05:05:00.772321 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.772326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.772433 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.772491 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.774392 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.775229 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.777506 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.777853 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.777927 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.783059 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.783062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.783064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.783093 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:00.784841 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.785555 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.785561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.785566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.785662 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:00.786124 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:00.786128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:00.786131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:00.786400 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:00.786507 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.787297 (+ 790us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.787448 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:05:00.789441 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.770720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.1.17.194:57339)
0807 05:05:00.771464 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.771481 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.772583 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.773021 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.773027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.773599 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.774224 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.774245 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.775421 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.775425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.777360 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.777371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.778143 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.778147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.778485 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.779344 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.779364 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.782839 (+ 3475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.785637 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.785641 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.785643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.785876 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.786203 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.786208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.786213 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.786299 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.786790 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.786793 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.786984 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.788013 (+ 1029us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.788368 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":1034,"spinlock_wait_cycles":18048,"thread_start_us":172,"threads_started":1}
I20260807 05:05:00.789461 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.770795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.194:42161)
0807 05:05:00.771560 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.771572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.772091 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.772842 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.772848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.773608 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.774225 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.774245 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.775421 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.775425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.777360 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.777371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.778143 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.778147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.778483 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.779441 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.779456 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.782839 (+ 3383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.784976 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.784985 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.785006 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.785428 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.785782 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.785787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.785790 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.785997 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.786596 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.786605 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.786991 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.787821 (+ 830us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.788235 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":635,"spinlock_wait_cycles":13056,"thread_start_us":142,"threads_started":1}
I20260807 05:05:00.790235 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.770421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.1.17.194:46401)
0807 05:05:00.771054 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.771105 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.772074 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.772834 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.772844 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.773408 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.774221 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.774243 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.775459 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.775465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.777360 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.777375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.778234 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.778241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.778480 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.779289 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.779315 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.782938 (+ 3623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:00.785442 (+ 2504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.785448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.785450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.785694 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.786035 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:00.786041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:00.786045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:00.786130 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:00.786624 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:00.786627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:00.786984 (+ 357us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.788002 (+ 1018us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.788309 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":505,"spinlock_wait_cycles":13568,"thread_start_us":138,"threads_started":1}
I20260807 05:05:00.791100 8623 heartbeater.cc:347] Connected to a master server at 127.1.17.253:39489
I20260807 05:05:00.791368 8621 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35215
I20260807 05:05:00.791373 8623 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:00.791448 8621 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:00.791589 8622 heartbeater.cc:347] Connected to a master server at 127.1.17.252:36303
I20260807 05:05:00.791673 8622 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:00.792037 8623 heartbeater.cc:511] Master 127.1.17.253:39489 requested a full tablet report, sending...
I20260807 05:05:00.792040 8622 heartbeater.cc:511] Master 127.1.17.252:36303 requested a full tablet report, sending...
I20260807 05:05:00.792644 8621 heartbeater.cc:511] Master 127.1.17.254:35215 requested a full tablet report, sending...
I20260807 05:05:00.793233 7443 ts_manager.cc:194] Re-registered known tserver with Master: 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727)
I20260807 05:05:00.793284 8085 ts_manager.cc:194] Registered new tserver with Master: 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727)
I20260807 05:05:00.793316 8270 ts_manager.cc:194] Registered new tserver with Master: 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727)
I20260807 05:05:00.793955 7443 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:42161
I20260807 05:05:00.872592 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.869082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.0.0.1:41274)
0807 05:05:00.869236 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.869273 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.869409 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.869729 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.869733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.869756 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:00.869882 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.869887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.870691 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.870695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.871468 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.871477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.872194 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.872198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.872286 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.872309 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:05:00.872375 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 05:05:00.872451 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 05:05:00.872591 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.869206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.1.17.252:36303)
0807 05:05:00.869367 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.869372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.869389 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:05:00.869539 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.869542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.869608 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.869708 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:00.870014 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.870570 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.871601 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.872067 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.872187 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.872346 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:00.872387 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 05:05:00.872463 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:05:00.873433 8280 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 05:05:00.873857 8323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c4889c01ea4853ac4ec259fb340b20. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:00.874022 8323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: This master's current role is: LEARNER
W20260807 05:05:00.904095 8628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:00.904330 8628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:00.904367 8628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:00.908144 8628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:00.908192 8628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:00.908242 8628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:05:00.913491 8628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab
--principal=oryx/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:38633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=41087
--tserver_master_addrs=127.1.17.254:35215,127.1.17.253:39489,127.1.17.252:36303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:00.914769 8628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:00.916160 8628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:05:00.926231 8628 server_base.cc:1034] running on GCE node
W20260807 05:05:00.926183 8637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.926151 8635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:00.926162 8639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:00.926827 8628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:00.927384 8628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:00.928547 8628 hybrid_clock.cc:648] HybridClock initialized: now 1786079100928533 us; error 29 us; skew 500 ppm
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:00.931336 8628 init.cc:377] Logged in from keytab as oryx/127.1.17.195@KRBTEST.COM (short username oryx)
I20260807 05:05:00.932228 8628 webserver.cc:460] Webserver started at http://127.1.17.195:41087/ using document root <none> and password file <none>
I20260807 05:05:00.932729 8628 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:00.932857 8628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:00.936656 8628 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:05:00.938746 8646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.939970 8628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:00.940099 8628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "053d912d6fbf44cea636bdb2910c599a"
format_stamp: "Formatted at 2026-08-07 05:04:59 on dist-test-slave-1x4g"
server_key: "30e41d699fa4e181f1e1464caf520b99"
server_key_iv: "a025dad24ccb6286f4c35ce512df8c54"
server_key_version: "encryptionkey@0"
I20260807 05:05:00.940649 8628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:00.969070 8628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:00.972497 8628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:00.972740 8628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:00.973379 8628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:00.974293 8628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:00.974365 8628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.974443 8628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:00.974493 8628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:00.983876 8628 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:38633
I20260807 05:05:00.983901 8763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:38633 every 8 connection(s)
I20260807 05:05:00.985039 8628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:05:00.993031 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 8628
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.253@KRBTEST.COM
Aug 07 05:05:00 dist-test-slave-1x4g krb5kdc[7391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079100, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.252@KRBTEST.COM
I20260807 05:05:01.003264 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.988220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:44535 (local address 127.1.17.252:36303)
0807 05:05:00.988353 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.988358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.988521 (+ 163us) server_negotiation.cc:407] Connection header received
0807 05:05:00.989647 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.989650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.989724 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.989777 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.991758 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.992607 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.995823 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.996073 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.996177 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.999678 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.999683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.999688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.999727 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.001484 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.001981 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.001987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.001989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.002079 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.002379 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.002382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.002384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.002599 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.002684 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.002996 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.003112 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:05:01.003321 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.986850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:39305 (local address 127.1.17.253:39489)
0807 05:05:00.986995 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.986999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.988541 (+ 1542us) server_negotiation.cc:407] Connection header received
0807 05:05:00.989649 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.989653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.989723 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.989773 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.991809 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.992665 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.994713 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.995038 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.995102 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.999000 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.999004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.999007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.999033 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.000653 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.001321 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.001325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.001327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.001431 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.001884 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.001887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.001889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.002082 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.002197 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.002808 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.003118 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:05:01.003505 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.987660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:43929 (local address 127.1.17.254:35215)
0807 05:05:00.987772 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:00.987776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:00.989053 (+ 1277us) server_negotiation.cc:407] Connection header received
0807 05:05:00.989647 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:00.989650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:00.989723 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:00.989777 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:00.991758 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.992612 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.994730 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.995046 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.995120 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.999282 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:00.999286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:00.999289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:00.999314 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.001700 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.002166 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.002170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.002172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.002265 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.002564 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.002567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.002569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.002798 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.002889 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.003227 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.003353 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:05:01.004160 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.987968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.195:43929)
0807 05:05:00.988958 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.988970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.989433 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.990163 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.990172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.991091 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.991581 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.991594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.993036 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.993039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.994589 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.994600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.995308 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.995311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.995547 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.996257 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.996284 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.999147 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:01.001808 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:01.001811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:01.001815 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:01.002049 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:01.002367 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:01.002370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:01.002372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:01.002466 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:01.002939 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:01.002942 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:01.003032 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.003630 (+ 598us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.003757 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":168,"spinlock_wait_cycles":5760,"thread_start_us":92,"threads_started":1}
I20260807 05:05:01.004518 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.987310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.1.17.195:39305)
0807 05:05:00.988395 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.988410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.989414 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.990163 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.990171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.990769 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.991573 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.991591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.992894 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.992897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.994584 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.994594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.995306 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.995310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.995544 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.996363 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.996378 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.998798 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:01.000756 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:01.000763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:01.000804 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:01.001190 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:01.001551 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:01.001557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:01.001560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:01.001739 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:01.002198 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:01.002204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:01.002517 (+ 313us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.002731 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.003171 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":452,"spinlock_wait_cycles":19712,"thread_start_us":173,"threads_started":1}
I20260807 05:05:01.005021 8770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:00.987310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.1.17.195:44535)
0807 05:05:00.988347 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:00.988401 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:00.989433 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:00.990163 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:00.990172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:00.990769 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:00.991578 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.991594 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.994856 (+ 3262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.994859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.995671 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:00.995682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:00.996348 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:00.996354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:00.996458 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:00.997005 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:00.997021 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:00.999521 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:01.001613 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:01.001616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:01.001618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:01.001860 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:01.002192 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:01.002195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:01.002196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:01.002276 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:01.002741 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:01.002745 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:01.002823 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.004052 (+ 1229us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.004830 (+ 778us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":57,"spinlock_wait_cycles":12800,"thread_start_us":147,"threads_started":1}
I20260807 05:05:01.005498 8764 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35215
I20260807 05:05:01.005748 8764 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:01.005851 8765 heartbeater.cc:347] Connected to a master server at 127.1.17.252:36303
I20260807 05:05:01.005921 8765 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:01.006197 8766 heartbeater.cc:347] Connected to a master server at 127.1.17.253:39489
I20260807 05:05:01.006352 8765 heartbeater.cc:511] Master 127.1.17.252:36303 requested a full tablet report, sending...
I20260807 05:05:01.006433 8766 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:01.006577 8766 heartbeater.cc:511] Master 127.1.17.253:39489 requested a full tablet report, sending...
I20260807 05:05:01.007431 8270 ts_manager.cc:194] Registered new tserver with Master: 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633)
I20260807 05:05:01.007431 8085 ts_manager.cc:194] Registered new tserver with Master: 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633)
I20260807 05:05:01.007879 8764 heartbeater.cc:511] Master 127.1.17.254:35215 requested a full tablet report, sending...
I20260807 05:05:01.008538 7443 ts_manager.cc:194] Re-registered known tserver with Master: 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633)
I20260807 05:05:01.010128 7443 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:43929
I20260807 05:05:01.017083 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.006778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.1.17.252:36303)
0807 05:05:01.007880 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.007884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.008503 (+ 619us) server_negotiation.cc:407] Connection header received
0807 05:05:01.008678 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.008681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.008719 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.008987 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:01.009707 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.010177 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.011395 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.011692 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.011786 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.013553 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:01.013557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:01.013560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:01.013590 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.015277 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.015775 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.015778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.015781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.015871 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.016228 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.016231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.016233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.016443 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.016573 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.016765 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.016921 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033}
I20260807 05:05:01.017092 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.005762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.1.17.253:39489)
0807 05:05:01.005914 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.005918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.007165 (+ 1247us) server_negotiation.cc:407] Connection header received
0807 05:05:01.007363 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.007367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.007433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.007486 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:01.008851 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.009613 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.010712 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.011032 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.011103 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.012991 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:01.012995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:01.012997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:01.013028 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.015145 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.015587 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.015590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.015592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.015677 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.016025 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.016028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.016030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.016265 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.016417 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.016798 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.016915 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 05:05:01.017876 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.006284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.1.17.254:35215)
0807 05:05:01.006446 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.006451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.007189 (+ 738us) server_negotiation.cc:407] Connection header received
0807 05:05:01.007580 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.007586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.007636 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.007686 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:01.008851 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.009607 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.011877 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.012266 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.012357 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.014281 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:01.014285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:01.014287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:01.014314 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.016024 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.016465 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.016470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.016473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.016561 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.016892 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.016897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.016899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.017098 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.017202 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.017623 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.017738 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260807 05:05:01.020587 7443 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:05:01.023630 7443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:01.026147 8781 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: NON_VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: true } } }
I20260807 05:05:01.026295 8781 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:01.029532 8243 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: attempting to promote NON_VOTER 547729d48f9b447a982c25d1e8b645b7 to VOTER
I20260807 05:05:01.029865 8243 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } }
I20260807 05:05:01.031013 8280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 LEARNER]: Refusing update from remote peer a3c4889c01ea4853ac4ec259fb340b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 05:05:01.031695 8780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 05:05:01.031754 8096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Refusing update from remote peer a3c4889c01ea4853ac4ec259fb340b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 05:05:01.032127 8780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 05:05:01.033365 8243 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.034149 8096 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.034878 8780 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.035014 8780 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:01.035072 8781 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.035166 8781 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:01.034893 8243 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.035641 8243 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:01.035815 8280 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.036545 8790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.036660 8790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:05:01.041296 8323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3c4889c01ea4853ac4ec259fb340b20" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:01.041416 8323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:05:01.044160 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.038928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.1.17.194:39727)
0807 05:05:01.039301 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.039314 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.039339 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:05:01.039474 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.039479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.039663 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.039827 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:01.040044 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.041175 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.042703 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.043524 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.043625 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.043792 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.043896 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.044010 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":117,"threads_started":1}
I20260807 05:05:01.044121 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.038731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:39727 (local address 127.0.0.1:58514)
0807 05:05:01.039259 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:01.039271 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:01.039351 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:01.039806 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:01.039809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:01.039827 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:01.039946 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.039955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.041307 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.041310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.042546 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.042559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.043650 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.043653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.043738 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.043783 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.043907 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.043983 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":113,"threads_started":1}
I20260807 05:05:01.045553 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.038491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:38633 (local address 127.0.0.1:60134)
0807 05:05:01.038974 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:01.038987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:01.039060 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:01.040332 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:01.040335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:01.040354 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:01.040444 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.040450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.042564 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.042566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.043690 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.043700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.045094 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.045097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.045249 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.045272 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.045348 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.045418 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":1}
I20260807 05:05:01.045917 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.039532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.1.17.195:38633)
0807 05:05:01.039832 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.039845 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.039869 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:05:01.039957 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.039963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.040176 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.040339 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:01.040539 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.042464 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.043806 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.044924 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.045098 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.045279 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.045628 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.045779 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":124,"threads_started":1}
I20260807 05:05:01.047302 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.039072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:43031 (local address 127.0.0.1:33706)
0807 05:05:01.039308 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:01.039320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:01.039381 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:01.041084 (+ 1703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:01.041087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:01.041102 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:01.041174 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.041180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.044011 (+ 2831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.044014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.045846 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.045856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.046920 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.046924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.047018 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.047040 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.047122 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.047186 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":620,"thread_start_us":76,"threads_started":1}
I20260807 05:05:01.047382 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.040030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.1.17.193:43031)
0807 05:05:01.040619 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.040634 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.040657 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:05:01.040732 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.040737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.040941 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.041323 (+ 382us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:01.041462 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.043884 (+ 2422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.046030 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.046796 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.046919 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.047049 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.047157 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.047240 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":98,"threads_started":1}
I20260807 05:05:01.048614 8552 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2fe20ee3714307a71142d056f9416f (DEFAULT_TABLE table=test-table [id=37fb64cf0b154b0e8f89273eb62c953b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:01.048557 8695 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2fe20ee3714307a71142d056f9416f (DEFAULT_TABLE table=test-table [id=37fb64cf0b154b0e8f89273eb62c953b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:01.049759 8552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2fe20ee3714307a71142d056f9416f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:01.049765 8695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2fe20ee3714307a71142d056f9416f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:01.051723 8400 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2fe20ee3714307a71142d056f9416f (DEFAULT_TABLE table=test-table [id=37fb64cf0b154b0e8f89273eb62c953b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:01.052696 8400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2fe20ee3714307a71142d056f9416f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:01.056828 8797 tablet_bootstrap.cc:492] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Bootstrap starting.
I20260807 05:05:01.058259 8798 tablet_bootstrap.cc:492] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Bootstrap starting.
I20260807 05:05:01.058998 8797 tablet_bootstrap.cc:654] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:01.059752 8797 log.cc:824] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:01.060385 8798 tablet_bootstrap.cc:654] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:01.061414 8798 log.cc:824] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:01.061535 8801 tablet_bootstrap.cc:492] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Bootstrap starting.
I20260807 05:05:01.062507 8797 tablet_bootstrap.cc:492] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: No bootstrap required, opened a new log
I20260807 05:05:01.062690 8797 ts_tablet_manager.cc:1397] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:05:01.063316 8798 tablet_bootstrap.cc:492] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: No bootstrap required, opened a new log
I20260807 05:05:01.063457 8798 ts_tablet_manager.cc:1397] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:05:01.063601 8801 tablet_bootstrap.cc:654] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:01.064380 8801 log.cc:824] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:01.066416 8801 tablet_bootstrap.cc:492] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: No bootstrap required, opened a new log
I20260807 05:05:01.066639 8801 ts_tablet_manager.cc:1397] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:05:01.067133 8798 raft_consensus.cc:348] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.067395 8798 raft_consensus.cc:374] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:01.067459 8798 raft_consensus.cc:729] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6696fca0c7dc4d709fd6621f51d20d0f, State: Initialized, Role: FOLLOWER
I20260807 05:05:01.067939 8798 consensus_queue.cc:260] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.067893 8797 raft_consensus.cc:348] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.068137 8797 raft_consensus.cc:374] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:01.068194 8797 raft_consensus.cc:729] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053d912d6fbf44cea636bdb2910c599a, State: Initialized, Role: FOLLOWER
I20260807 05:05:01.068594 8621 heartbeater.cc:503] Master 127.1.17.254:35215 was elected leader, sending a full tablet report...
I20260807 05:05:01.068632 8797 consensus_queue.cc:260] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.069591 8797 ts_tablet_manager.cc:1428] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 05:05:01.069773 8764 heartbeater.cc:503] Master 127.1.17.254:35215 was elected leader, sending a full tablet report...
I20260807 05:05:01.073077 8798 ts_tablet_manager.cc:1428] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Time spent starting tablet: real 0.010s user 0.002s sys 0.005s
I20260807 05:05:01.073503 8801 raft_consensus.cc:348] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.073760 8801 raft_consensus.cc:374] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:01.073809 8801 raft_consensus.cc:729] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9149d2f3864246beb77be7325bd135, State: Initialized, Role: FOLLOWER
I20260807 05:05:01.074218 8801 consensus_queue.cc:260] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.075203 8801 ts_tablet_manager.cc:1428] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260807 05:05:01.075860 8475 heartbeater.cc:503] Master 127.1.17.254:35215 was elected leader, sending a full tablet report...
W20260807 05:05:01.236938 8767 tablet.cc:2369] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:05:01.269737 8624 tablet.cc:2369] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:05:01.311544 8478 tablet.cc:2369] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:05:01.453389 8805 raft_consensus.cc:482] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:01.453610 8805 raft_consensus.cc:504] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.454777 8805 leader_election.cc:288] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727), da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031)
I20260807 05:05:01.458537 8804 raft_consensus.cc:482] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:01.458729 8804 raft_consensus.cc:504] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.458990 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.455592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:41501 (local address 127.1.17.193:43031)
0807 05:05:01.455720 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.455723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.455736 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:01.455782 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.455785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.455823 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.455935 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:01.456240 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.456847 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.457974 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.458378 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.458499 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.458551 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.458798 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.458874 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:05:01.458982 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.455450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:43031 (local address 127.1.17.195:41501)
0807 05:05:01.455562 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:01.455578 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:01.455652 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:01.455945 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:01.455948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:01.455967 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:01.456119 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.456124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.456962 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.456967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.457840 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.457849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.458492 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.458496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.458599 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.458624 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.458710 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.458789 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":43}
I20260807 05:05:01.458995 8770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.455099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:39727 (local address 127.1.17.195:55911)
0807 05:05:01.455297 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:01.455312 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:01.455424 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:01.455910 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:01.455922 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:01.455938 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:01.456105 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.456111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.456966 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.456969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.457841 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.457850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.458578 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.458582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.458688 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.458713 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.458768 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.458832 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":49}
I20260807 05:05:01.459370 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.455179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:55911 (local address 127.1.17.194:39727)
0807 05:05:01.455351 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.455359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.455379 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:05:01.455760 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.455763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.455808 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.455890 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:01.456227 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.456851 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.457983 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.458435 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.459084 (+ 649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.459135 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.459189 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.459256 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:05:01.460047 8804 leader_election.cc:288] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633), da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031)
I20260807 05:05:01.460134 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2fe20ee3714307a71142d056f9416f" candidate_uuid: "053d912d6fbf44cea636bdb2910c599a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da9149d2f3864246beb77be7325bd135" is_pre_election: true
I20260807 05:05:01.460143 8573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2fe20ee3714307a71142d056f9416f" candidate_uuid: "053d912d6fbf44cea636bdb2910c599a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" is_pre_election: true
I20260807 05:05:01.460354 8573 raft_consensus.cc:2454] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 053d912d6fbf44cea636bdb2910c599a in term 0.
I20260807 05:05:01.460430 8421 raft_consensus.cc:2454] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 053d912d6fbf44cea636bdb2910c599a in term 0.
I20260807 05:05:01.460975 8650 leader_election.cc:302] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053d912d6fbf44cea636bdb2910c599a, 6696fca0c7dc4d709fd6621f51d20d0f; no voters:
I20260807 05:05:01.461278 8805 raft_consensus.cc:2781] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:01.461444 8805 raft_consensus.cc:482] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:01.461524 8805 raft_consensus.cc:3037] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:01.462656 8805 raft_consensus.cc:504] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.462984 8805 leader_election.cc:288] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [CANDIDATE]: Term 1 election: Requested vote from peers 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727), da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031)
I20260807 05:05:01.463338 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2fe20ee3714307a71142d056f9416f" candidate_uuid: "053d912d6fbf44cea636bdb2910c599a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da9149d2f3864246beb77be7325bd135"
I20260807 05:05:01.463349 8573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2fe20ee3714307a71142d056f9416f" candidate_uuid: "053d912d6fbf44cea636bdb2910c599a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6696fca0c7dc4d709fd6621f51d20d0f"
I20260807 05:05:01.463452 8421 raft_consensus.cc:3037] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:01.463485 8573 raft_consensus.cc:3037] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:01.464509 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.460273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:36879 (local address 127.1.17.195:38633)
0807 05:05:01.460658 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.460663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.460677 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:01.460718 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.460723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.460767 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.460859 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:01.461352 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.462000 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.463316 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.463870 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.463954 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.463992 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.464312 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.464380 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260807 05:05:01.464643 8421 raft_consensus.cc:2454] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053d912d6fbf44cea636bdb2910c599a in term 1.
I20260807 05:05:01.464763 8573 raft_consensus.cc:2454] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 053d912d6fbf44cea636bdb2910c599a in term 1.
I20260807 05:05:01.464979 8650 leader_election.cc:302] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053d912d6fbf44cea636bdb2910c599a, da9149d2f3864246beb77be7325bd135; no voters:
I20260807 05:05:01.465205 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.460192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:38633 (local address 127.1.17.194:36879)
0807 05:05:01.460435 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:01.460452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:01.460550 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:01.461133 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:01.461136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:01.461149 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:01.461234 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.461239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.462150 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.462153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.463191 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.463201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.464080 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.464083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.464180 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.464202 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.464628 (+ 426us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.464727 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260807 05:05:01.465250 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.461048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:36901 (local address 127.1.17.193:43031)
0807 05:05:01.461193 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.461196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.461220 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:05:01.461263 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.461265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.461308 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.461380 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:01.461747 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.462402 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.463709 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.464206 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.464304 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.464345 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.464727 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.465111 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 05:05:01.465453 8805 raft_consensus.cc:2781] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:01.465613 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.460537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:43031 (local address 127.1.17.194:36901)
0807 05:05:01.460934 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:01.460947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:01.461029 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:01.461482 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:01.461485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:01.461506 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:01.461626 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.461633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.462522 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.462528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.463568 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:01.463579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.464442 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.464446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:01.464570 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.464605 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 05:05:01.465398 (+ 793us) client_negotiation.cc:240] Negotiation successful
0807 05:05:01.465477 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":52,"threads_started":1}
I20260807 05:05:01.465641 8805 raft_consensus.cc:686] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 1 LEADER]: Becoming Leader. State: Replica: 053d912d6fbf44cea636bdb2910c599a, State: Running, Role: LEADER
I20260807 05:05:01.465747 8716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2fe20ee3714307a71142d056f9416f" candidate_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053d912d6fbf44cea636bdb2910c599a" is_pre_election: true
I20260807 05:05:01.466293 8805 consensus_queue.cc:237] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:01.466634 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2fe20ee3714307a71142d056f9416f" candidate_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da9149d2f3864246beb77be7325bd135" is_pre_election: true
I20260807 05:05:01.466814 8421 raft_consensus.cc:2379] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6696fca0c7dc4d709fd6621f51d20d0f in current term 1: Already voted for candidate 053d912d6fbf44cea636bdb2910c599a in this term.
I20260807 05:05:01.467342 8504 leader_election.cc:302] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6696fca0c7dc4d709fd6621f51d20d0f; no voters: 053d912d6fbf44cea636bdb2910c599a, da9149d2f3864246beb77be7325bd135
I20260807 05:05:01.467592 8804 raft_consensus.cc:2726] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 05:05:01.470095 7443 catalog_manager.cc:5495] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a reported cstate change: term changed from 0 to 1, leader changed from <none> to 053d912d6fbf44cea636bdb2910c599a (127.1.17.195). New cstate: current_term: 1 leader_uuid: "053d912d6fbf44cea636bdb2910c599a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260807 05:05:01.493469 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.489768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60140 (local address 127.1.17.195:38633)
0807 05:05:01.489915 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.489920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.489934 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:01.490088 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.490090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.490151 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.490257 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:01.490507 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.491087 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.492012 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.492221 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.492305 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.492606 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:01.493087 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:01.493183 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.493236 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.493313 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 05:05:01.499388 8573 raft_consensus.cc:1261] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Refusing update from remote peer 053d912d6fbf44cea636bdb2910c599a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:01.499351 8421 raft_consensus.cc:1261] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 1 FOLLOWER]: Refusing update from remote peer 053d912d6fbf44cea636bdb2910c599a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:01.500128 8809 consensus_queue.cc:1035] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:01.500340 8805 consensus_queue.cc:1035] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [LEADER]: Connected to new peer: Peer: permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:01.508841 8812 mvcc.cc:205] Tried to move back new op lower bound from 7315779999734001664 to 7315779999608856576. Current Snapshot: MvccSnapshot[applied={T|T < 7315779999734001664}]
I20260807 05:05:01.513759 8813 mvcc.cc:205] Tried to move back new op lower bound from 7315779999734001664 to 7315779999608856576. Current Snapshot: MvccSnapshot[applied={T|T < 7315779999734001664}]
I20260807 05:05:01.518327 7439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:01.518465 7439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:01.524082 7439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3c4889c01ea4853ac4ec259fb340b20: Sending DeleteTablet for 3 replicas of tablet 3d2fe20ee3714307a71142d056f9416f
I20260807 05:05:01.524849 8695 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d2fe20ee3714307a71142d056f9416f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:01 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60134
I20260807 05:05:01.525116 8552 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d2fe20ee3714307a71142d056f9416f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:01 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:58514
I20260807 05:05:01.525103 8400 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d2fe20ee3714307a71142d056f9416f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:01 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:33706
I20260807 05:05:01.525393 8822 tablet_replica.cc:330] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: stopping tablet replica
I20260807 05:05:01.525718 8822 raft_consensus.cc:2229] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:05:01.526124 8822 raft_consensus.cc:2258] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:01.526176 8823 tablet_replica.cc:330] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: stopping tablet replica
I20260807 05:05:01.526484 8823 raft_consensus.cc:2229] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:01.526803 8823 raft_consensus.cc:2258] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:01.527130 8824 tablet_replica.cc:330] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: stopping tablet replica
I20260807 05:05:01.527406 8824 raft_consensus.cc:2229] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:01.527661 8824 raft_consensus.cc:2258] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:01.527714 8822 ts_tablet_manager.cc:1905] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:01.528577 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 7406
I20260807 05:05:01.531137 8822 ts_tablet_manager.cc:1918] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:01.531281 8822 log.cc:1196] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/3d2fe20ee3714307a71142d056f9416f
I20260807 05:05:01.531636 8822 ts_tablet_manager.cc:1939] T 3d2fe20ee3714307a71142d056f9416f P 053d912d6fbf44cea636bdb2910c599a: Deleting consensus metadata
I20260807 05:05:01.533809 8824 ts_tablet_manager.cc:1905] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:01.536506 8824 ts_tablet_manager.cc:1918] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:01.536621 8824 log.cc:1196] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/3d2fe20ee3714307a71142d056f9416f
I20260807 05:05:01.536967 8824 ts_tablet_manager.cc:1939] T 3d2fe20ee3714307a71142d056f9416f P da9149d2f3864246beb77be7325bd135: Deleting consensus metadata
W20260807 05:05:01.538226 8349 connection.cc:441] server connection from 127.0.0.1:33706 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33706 (request call id 1) could send its response
W20260807 05:05:01.538300 8647 connection.cc:535] server connection from 127.0.0.1:60134 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 05:05:01.538976 8823 ts_tablet_manager.cc:1905] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:01.541623 8823 ts_tablet_manager.cc:1918] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:01.541720 8823 log.cc:1196] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/3d2fe20ee3714307a71142d056f9416f
I20260807 05:05:01.542022 8823 ts_tablet_manager.cc:1939] T 3d2fe20ee3714307a71142d056f9416f P 6696fca0c7dc4d709fd6621f51d20d0f: Deleting consensus metadata
W20260807 05:05:01.543051 8504 connection.cc:441] server connection from 127.0.0.1:58514 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58514 (request call id 1) could send its response
I20260807 05:05:01.553560 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.541871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.1.17.253:39489)
0807 05:05:01.543613 (+ 1742us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.543617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.543629 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:05:01.543662 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.543664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.543699 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.543774 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:01.544763 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.545666 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.547074 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.547465 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.547561 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.549370 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:01.549375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:01.549377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:01.549405 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.551398 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.551880 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.551884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.551886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.551971 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.552251 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.552254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.552255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.552495 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.552623 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.553241 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.553364 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1679,"thread_start_us":86,"threads_started":1}
I20260807 05:05:01.554415 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.528143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.1.17.253:39489)
0807 05:05:01.542354 (+ 14211us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.542359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.542377 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:05:01.542429 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.542431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.542482 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.542903 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:01.544051 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.544950 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.546174 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.546516 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.546610 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.549192 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:01.549197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:01.549200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:01.549273 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.551811 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.552263 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.552268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.552271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.552388 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.552688 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.552691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.552692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.552948 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.553058 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.554089 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.554222 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14132,"thread_start_us":74,"threads_started":1}
I20260807 05:05:01.555037 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.541835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.1.17.252:36303)
0807 05:05:01.543215 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.543220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.543236 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:05:01.543278 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.543281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.543324 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.543437 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:01.544265 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.545129 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.546288 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.546684 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.546779 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.549357 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:01.549360 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:01.549363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:01.549389 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.551583 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.552092 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.552096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.552098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.552200 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.552497 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.552500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.552502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.552732 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.552921 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.553700 (+ 779us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.553825 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1303,"thread_start_us":52,"threads_started":1}
I20260807 05:05:01.563211 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:01.527724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41302 (local address 127.1.17.252:36303)
0807 05:05:01.543149 (+ 15425us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:01.543153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:01.553635 (+ 10482us) server_negotiation.cc:407] Connection header received
0807 05:05:01.554021 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:01.554025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:01.554069 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:01.554149 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:01.554997 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.555989 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.556754 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:01.557033 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:01.557134 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:01.558683 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:01.558686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:01.558689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:01.558712 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:01.561410 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.561927 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.561931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.561933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.562026 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:01.562335 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:01.562340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:01.562341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:01.562574 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:01.562702 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:01.562930 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 05:05:01.563055 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15350,"thread_start_us":91,"threads_started":1}
I20260807 05:05:02.514961 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:02.514238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:52431)
0807 05:05:02.514728 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:02.514827 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":68,"threads_started":1}
I20260807 05:05:02.515018 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:02.513816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.194:60203)
0807 05:05:02.514814 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:02.514913 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":923,"thread_start_us":82,"threads_started":1}
W20260807 05:05:02.515285 8621 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:35215 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
W20260807 05:05:02.515280 8475 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:35215 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
I20260807 05:05:02.517925 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:02.517571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.195:43549)
0807 05:05:02.517778 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:02.517844 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
W20260807 05:05:02.518190 8764 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:35215 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
I20260807 05:05:03.023497 8842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3c4889c01ea4853ac4ec259fb340b20)
I20260807 05:05:03.023643 8842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } }
I20260807 05:05:03.024307 8843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3c4889c01ea4853ac4ec259fb340b20)
I20260807 05:05:03.024592 8842 leader_election.cc:288] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215), 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252:36303)
I20260807 05:05:03.024539 8843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } }
I20260807 05:05:03.025020 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.024341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42132)
0807 05:05:03.024803 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.024902 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":27,"thread_start_us":68,"threads_started":1}
W20260807 05:05:03.025233 8071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
I20260807 05:05:03.026077 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.025842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42134)
0807 05:05:03.025958 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.026012 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 05:05:03.026188 8071 leader_election.cc:334] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215): Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
I20260807 05:05:03.026242 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.025863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42148)
0807 05:05:03.026069 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.026152 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":51,"threads_started":1}
W20260807 05:05:03.026522 8264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
I20260807 05:05:03.026850 8843 leader_election.cc:288] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215), 64e07cde7c3f4ed295fcdac69c507145 (127.1.17.253:39489)
I20260807 05:05:03.027519 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.027287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42152)
0807 05:05:03.027397 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.027451 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 05:05:03.027702 8264 leader_election.cc:334] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215): Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
I20260807 05:05:03.028959 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.024530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.1.17.252:36303)
0807 05:05:03.024924 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.024929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.024946 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:05:03.025000 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.025003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.025049 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.025146 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:03.025507 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.026466 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.027977 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.028402 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.028522 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.028583 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.028745 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.028838 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":71,"threads_started":1}
I20260807 05:05:03.028987 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.024341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36303 (local address 127.0.0.1:41326)
0807 05:05:03.024696 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.024713 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:03.024849 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:03.025238 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:03.025243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:03.025268 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:03.025381 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.025388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.026616 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.026620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.027848 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.027862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.028524 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.028527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.028617 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.028640 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:05:03.028729 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 05:05:03.028824 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260807 05:05:03.029511 8280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e07cde7c3f4ed295fcdac69c507145" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "547729d48f9b447a982c25d1e8b645b7" is_pre_election: true
I20260807 05:05:03.029699 8280 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64e07cde7c3f4ed295fcdac69c507145 in term 1.
I20260807 05:05:03.030037 8072 leader_election.cc:302] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 547729d48f9b447a982c25d1e8b645b7, 64e07cde7c3f4ed295fcdac69c507145; no voters: a3c4889c01ea4853ac4ec259fb340b20
I20260807 05:05:03.030181 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.026273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:39489 (local address 127.0.0.1:38252)
0807 05:05:03.026386 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.026401 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:03.026508 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:03.026949 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:03.026952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:03.026971 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:03.027078 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.027085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.028053 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.028057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.028920 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.028929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.029761 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.029765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.029861 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.029895 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 05:05:03.029975 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:05:03.030043 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260807 05:05:03.030194 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.026463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.1.17.253:39489)
0807 05:05:03.026748 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.026754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.026765 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:05:03.026800 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.026802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.026835 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.026933 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:03.027231 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.027944 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.029036 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.029580 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.029703 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.029754 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.030002 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.030068 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
I20260807 05:05:03.030241 8842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 05:05:03.030294 8842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3c4889c01ea4853ac4ec259fb340b20)
I20260807 05:05:03.030333 8842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:05:03.030587 8096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "547729d48f9b447a982c25d1e8b645b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "64e07cde7c3f4ed295fcdac69c507145" is_pre_election: true
I20260807 05:05:03.031462 8842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } }
I20260807 05:05:03.031694 8096 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 547729d48f9b447a982c25d1e8b645b7 in current term 2: Already voted for candidate 64e07cde7c3f4ed295fcdac69c507145 in this term.
I20260807 05:05:03.031834 8842 leader_election.cc:288] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 2 election: Requested vote from peers a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215), 547729d48f9b447a982c25d1e8b645b7 (127.1.17.252:36303)
I20260807 05:05:03.032146 8280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64e07cde7c3f4ed295fcdac69c507145" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "547729d48f9b447a982c25d1e8b645b7"
I20260807 05:05:03.032227 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.031955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42158)
0807 05:05:03.032109 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.032166 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 05:05:03.032284 8280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:05:03.032666 8263 leader_election.cc:302] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 547729d48f9b447a982c25d1e8b645b7; no voters: 64e07cde7c3f4ed295fcdac69c507145, a3c4889c01ea4853ac4ec259fb340b20
I20260807 05:05:03.032984 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.032692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42174)
0807 05:05:03.032880 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.032929 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 05:05:03.033115 8071 leader_election.cc:334] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215): Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
I20260807 05:05:03.033576 8280 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64e07cde7c3f4ed295fcdac69c507145 in term 2.
I20260807 05:05:03.033779 8843 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:05:03.033918 8072 leader_election.cc:302] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 547729d48f9b447a982c25d1e8b645b7, 64e07cde7c3f4ed295fcdac69c507145; no voters: a3c4889c01ea4853ac4ec259fb340b20
I20260807 05:05:03.034088 8842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:05:03.034266 8842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [term 2 LEADER]: Becoming Leader. State: Replica: 64e07cde7c3f4ed295fcdac69c507145, State: Running, Role: LEADER
I20260807 05:05:03.034518 8842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } }
I20260807 05:05:03.035786 8853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e07cde7c3f4ed295fcdac69c507145. Latest consensus state: current_term: 2 leader_uuid: "64e07cde7c3f4ed295fcdac69c507145" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:03.035907 8853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:03.036228 8855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:05:03.038414 8855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:05:03.038769 8855 catalog_manager.cc:1259] Loaded cluster ID: b59e5e2a51124417b8e7352683f68f51
I20260807 05:05:03.038811 8855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:05:03.039383 8855 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:05:03.039759 8855 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Loaded TSK: 0
I20260807 05:05:03.040426 8855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:05:03.066881 8086 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:05:03.070024 8086 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:03.072897 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.072585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42200)
0807 05:05:03.072709 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.072757 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 05:05:03.072947 8280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [term 2 FOLLOWER]: Refusing update from remote peer 64e07cde7c3f4ed295fcdac69c507145: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 05:05:03.073421 8853 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 05:05:03.073622 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.073392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.0.0.1:42216)
0807 05:05:03.073500 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.073549 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 05:05:03.073757 8071 consensus_peers.cc:487] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 -> Peer a3c4889c01ea4853ac4ec259fb340b20 (127.1.17.254:35215): Couldn't send request to peer a3c4889c01ea4853ac4ec259fb340b20. Status: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 05:05:03.074874 8843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e07cde7c3f4ed295fcdac69c507145. Latest consensus state: current_term: 2 leader_uuid: "64e07cde7c3f4ed295fcdac69c507145" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:03.075028 8843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:05:03.076125 8842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "64e07cde7c3f4ed295fcdac69c507145" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:03.076252 8842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:03.076524 8843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "64e07cde7c3f4ed295fcdac69c507145" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a3c4889c01ea4853ac4ec259fb340b20" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35215 } } peers { permanent_uuid: "64e07cde7c3f4ed295fcdac69c507145" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 39489 } attrs { promote: false } } peers { permanent_uuid: "547729d48f9b447a982c25d1e8b645b7" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36303 } attrs { promote: false } } }
I20260807 05:05:03.076660 8843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547729d48f9b447a982c25d1e8b645b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:05:03.076848 8857 mvcc.cc:205] Tried to move back new op lower bound from 7315780006182027264 to 7315780006032048128. Current Snapshot: MvccSnapshot[applied={T|T < 7315780006182027264}]
I20260807 05:05:03.086884 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.082687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:43031 (local address 127.0.0.1:33708)
0807 05:05:03.082991 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.083009 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:03.083086 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:03.083372 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:03.083377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:03.083393 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:03.083515 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.083525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.084382 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.084388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.085535 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.085545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.086420 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.086512 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.086553 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 05:05:03.086623 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:05:03.086701 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":114,"threads_started":1}
I20260807 05:05:03.087090 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.082251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:39727 (local address 127.0.0.1:58524)
0807 05:05:03.083228 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.083239 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:03.083310 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:03.083678 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:03.083680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:03.083696 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:03.083769 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.083774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.084751 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.084755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.085884 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.085894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.086610 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.086703 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.086727 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:05:03.086805 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 05:05:03.086876 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":16}
I20260807 05:05:03.087249 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.082481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:38633 (local address 127.0.0.1:60156)
0807 05:05:03.082995 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.083012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:03.083103 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:03.083540 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:03.083543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:03.083560 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:03.083669 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.083674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.084700 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.084704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.085952 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.085962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.086835 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.086950 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.086976 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 05:05:03.087054 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 05:05:03.087139 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":41}
I20260807 05:05:03.087393 8864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.082816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.1.17.193:43031)
0807 05:05:03.083062 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.083067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.083083 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:05:03.083186 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.083189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.083240 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.083362 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:03.083656 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.084272 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.085678 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.086297 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086400 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.086468 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.087168 (+ 700us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.087253 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":67,"threads_started":1}
I20260807 05:05:03.087634 8861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.082370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.1.17.194:39727)
0807 05:05:03.083284 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.083289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.083304 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:05:03.083430 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.083433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.083480 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.083559 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:03.083872 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.084628 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086012 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.086487 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086611 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.086676 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.087400 (+ 724us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.087506 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":90,"threads_started":1}
I20260807 05:05:03.087736 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.082621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60156 (local address 127.1.17.195:38633)
0807 05:05:03.083332 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.083338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.083352 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:03.083403 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.083406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.083451 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.083533 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:03.083787 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.084582 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086089 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.086720 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.086831 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.086891 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.087512 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.087605 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":84,"threads_started":1}
I20260807 05:05:03.087939 8400 tablet_service.cc:1467] Processing CreateTablet for tablet fcbabbaaa194494b8e76698d55282681 (DEFAULT_TABLE table=test-table [id=62703fac939641fcb069e604815d1349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:03.088205 8400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcbabbaaa194494b8e76698d55282681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:03.088212 8695 tablet_service.cc:1467] Processing CreateTablet for tablet fcbabbaaa194494b8e76698d55282681 (DEFAULT_TABLE table=test-table [id=62703fac939641fcb069e604815d1349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:03.088512 8695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcbabbaaa194494b8e76698d55282681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:03.089239 8552 tablet_service.cc:1467] Processing CreateTablet for tablet fcbabbaaa194494b8e76698d55282681 (DEFAULT_TABLE table=test-table [id=62703fac939641fcb069e604815d1349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:03.089509 8552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcbabbaaa194494b8e76698d55282681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:03.090973 8865 tablet_bootstrap.cc:492] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135: Bootstrap starting.
I20260807 05:05:03.091812 8865 tablet_bootstrap.cc:654] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:03.091961 8867 tablet_bootstrap.cc:492] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: Bootstrap starting.
I20260807 05:05:03.092408 8866 tablet_bootstrap.cc:492] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: Bootstrap starting.
I20260807 05:05:03.092860 8867 tablet_bootstrap.cc:654] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:03.093503 8866 tablet_bootstrap.cc:654] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:03.093780 8865 tablet_bootstrap.cc:492] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135: No bootstrap required, opened a new log
I20260807 05:05:03.093883 8865 ts_tablet_manager.cc:1397] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 05:05:03.094043 8867 tablet_bootstrap.cc:492] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: No bootstrap required, opened a new log
I20260807 05:05:03.094156 8867 ts_tablet_manager.cc:1397] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 05:05:03.094399 8865 raft_consensus.cc:348] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.094527 8865 raft_consensus.cc:374] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:03.094566 8865 raft_consensus.cc:729] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9149d2f3864246beb77be7325bd135, State: Initialized, Role: FOLLOWER
I20260807 05:05:03.094714 8866 tablet_bootstrap.cc:492] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: No bootstrap required, opened a new log
I20260807 05:05:03.094642 8867 raft_consensus.cc:348] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.094689 8865 consensus_queue.cc:260] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.094765 8867 raft_consensus.cc:374] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:03.094797 8867 raft_consensus.cc:729] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6696fca0c7dc4d709fd6621f51d20d0f, State: Initialized, Role: FOLLOWER
I20260807 05:05:03.094805 8866 ts_tablet_manager.cc:1397] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:03.094935 8867 consensus_queue.cc:260] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.095007 8865 ts_tablet_manager.cc:1428] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 05:05:03.095271 8867 ts_tablet_manager.cc:1428] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 05:05:03.095249 8866 raft_consensus.cc:348] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.095366 8866 raft_consensus.cc:374] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:03.095400 8866 raft_consensus.cc:729] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053d912d6fbf44cea636bdb2910c599a, State: Initialized, Role: FOLLOWER
I20260807 05:05:03.095563 8866 consensus_queue.cc:260] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.095882 8866 ts_tablet_manager.cc:1428] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:03.096903 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.095623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:50549)
0807 05:05:03.096680 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.096744 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":101,"threads_started":1}
I20260807 05:05:03.097735 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.095833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.194:55555)
0807 05:05:03.097558 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.097618 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1508,"thread_start_us":112,"threads_started":1}
I20260807 05:05:03.099198 8877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.096247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.195:44257)
0807 05:05:03.099062 (+ 2815us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.099118 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2751,"thread_start_us":91,"threads_started":1}
I20260807 05:05:03.099507 8623 heartbeater.cc:503] Master 127.1.17.253:39489 was elected leader, sending a full tablet report...
I20260807 05:05:03.099879 8766 heartbeater.cc:503] Master 127.1.17.253:39489 was elected leader, sending a full tablet report...
I20260807 05:05:03.101197 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.100542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:49531)
0807 05:05:03.101061 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.101122 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450}
I20260807 05:05:03.101388 8477 heartbeater.cc:503] Master 127.1.17.253:39489 was elected leader, sending a full tablet report...
I20260807 05:05:03.101586 8877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.100009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.195:46781)
0807 05:05:03.101424 (+ 1415us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.101527 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1345}
I20260807 05:05:03.102036 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.099674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.194:54537)
0807 05:05:03.101907 (+ 2233us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.101969 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2151}
I20260807 05:05:03.146919 8871 raft_consensus.cc:482] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:03.147074 8871 raft_consensus.cc:504] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.148250 8871 leader_election.cc:288] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727), 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633)
I20260807 05:05:03.152168 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.148459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:39727 (local address 127.1.17.193:38413)
0807 05:05:03.148592 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.148609 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:03.148709 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:03.149084 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:03.149087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:03.149105 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:03.149218 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.149222 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.150191 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.150194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.151030 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.151040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.151727 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.151731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.151847 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.151866 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:05:03.151962 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 05:05:03.152031 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 05:05:03.152168 8861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.148603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:38413 (local address 127.1.17.194:39727)
0807 05:05:03.148728 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.148734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.148754 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:05:03.148846 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.148877 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.148965 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.149072 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:03.149321 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.150083 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.151165 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.151582 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.151719 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.151778 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.151973 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.152054 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 05:05:03.152662 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.148729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:38633 (local address 127.1.17.193:36395)
0807 05:05:03.149363 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.149373 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:03.149437 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:03.149682 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:03.149684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:03.149712 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:03.149824 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.149829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.150727 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.150732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.151504 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:03.151514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.152184 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.152188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:03.152298 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.152328 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 05:05:03.152421 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 05:05:03.152492 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":193,"threads_started":1}
I20260807 05:05:03.152683 8573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcbabbaaa194494b8e76698d55282681" candidate_uuid: "da9149d2f3864246beb77be7325bd135" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" is_pre_election: true
I20260807 05:05:03.152865 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.148881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:36395 (local address 127.1.17.195:38633)
0807 05:05:03.149025 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.149032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.149452 (+ 420us) server_negotiation.cc:407] Connection header received
0807 05:05:03.149521 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.149524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.149582 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.149675 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:03.149946 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.150594 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.151642 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.152055 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.152193 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.152246 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.152602 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.152689 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:05:03.152891 8573 raft_consensus.cc:2454] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da9149d2f3864246beb77be7325bd135 in term 0.
I20260807 05:05:03.153357 8351 leader_election.cc:302] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6696fca0c7dc4d709fd6621f51d20d0f, da9149d2f3864246beb77be7325bd135; no voters:
I20260807 05:05:03.153462 8716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcbabbaaa194494b8e76698d55282681" candidate_uuid: "da9149d2f3864246beb77be7325bd135" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053d912d6fbf44cea636bdb2910c599a" is_pre_election: true
I20260807 05:05:03.153658 8871 raft_consensus.cc:2781] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:03.153714 8871 raft_consensus.cc:482] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:03.153690 8716 raft_consensus.cc:2454] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da9149d2f3864246beb77be7325bd135 in term 0.
I20260807 05:05:03.153743 8871 raft_consensus.cc:3037] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:03.154866 8871 raft_consensus.cc:504] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.155187 8871 leader_election.cc:288] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [CANDIDATE]: Term 1 election: Requested vote from peers 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727), 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633)
I20260807 05:05:03.155556 8573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcbabbaaa194494b8e76698d55282681" candidate_uuid: "da9149d2f3864246beb77be7325bd135" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6696fca0c7dc4d709fd6621f51d20d0f"
I20260807 05:05:03.155691 8573 raft_consensus.cc:3037] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:03.155645 8716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcbabbaaa194494b8e76698d55282681" candidate_uuid: "da9149d2f3864246beb77be7325bd135" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053d912d6fbf44cea636bdb2910c599a"
I20260807 05:05:03.155789 8716 raft_consensus.cc:3037] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:03.156634 8573 raft_consensus.cc:2454] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da9149d2f3864246beb77be7325bd135 in term 1.
I20260807 05:05:03.156925 8716 raft_consensus.cc:2454] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da9149d2f3864246beb77be7325bd135 in term 1.
I20260807 05:05:03.156924 8351 leader_election.cc:302] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6696fca0c7dc4d709fd6621f51d20d0f, da9149d2f3864246beb77be7325bd135; no voters:
I20260807 05:05:03.157218 8871 raft_consensus.cc:2781] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:03.157469 8871 raft_consensus.cc:686] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 1 LEADER]: Becoming Leader. State: Replica: da9149d2f3864246beb77be7325bd135, State: Running, Role: LEADER
I20260807 05:05:03.157747 8871 consensus_queue.cc:237] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } }
I20260807 05:05:03.158390 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.158140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:44257)
0807 05:05:03.158256 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.158317 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 05:05:03.160346 8086 catalog_manager.cc:5495] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 reported cstate change: term changed from 0 to 1, leader changed from <none> to da9149d2f3864246beb77be7325bd135 (127.1.17.193). New cstate: current_term: 1 leader_uuid: "da9149d2f3864246beb77be7325bd135" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da9149d2f3864246beb77be7325bd135" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260807 05:05:03.185829 8864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.182493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33722 (local address 127.1.17.193:43031)
0807 05:05:03.182622 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:03.182625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:03.182639 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:03.182718 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:03.182720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:03.182779 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:03.182879 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:03.183111 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.183627 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.184448 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:03.184653 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:03.184728 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:03.185006 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:03.185483 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:03.185573 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:03.185657 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 05:05:03.185711 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:05:03.188825 8573 raft_consensus.cc:1261] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Refusing update from remote peer da9149d2f3864246beb77be7325bd135: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:03.189042 8716 raft_consensus.cc:1261] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 1 FOLLOWER]: Refusing update from remote peer da9149d2f3864246beb77be7325bd135: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:03.189389 8871 consensus_queue.cc:1035] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6696fca0c7dc4d709fd6621f51d20d0f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 39727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:03.189615 8879 consensus_queue.cc:1035] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [LEADER]: Connected to new peer: Peer: permanent_uuid: "053d912d6fbf44cea636bdb2910c599a" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:03.191638 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.191243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.194:38195)
0807 05:05:03.191390 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.191457 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 05:05:03.191991 8877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.191710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.195:34649)
0807 05:05:03.191848 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.191910 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 05:05:03.194072 8883 mvcc.cc:205] Tried to move back new op lower bound from 7315780006655684608 to 7315780006537261056. Current Snapshot: MvccSnapshot[applied={T|T < 7315780006655684608}]
I20260807 05:05:03.195425 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.195195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:33513)
0807 05:05:03.195295 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.195356 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 05:05:03.196244 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.196003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.193:53807)
0807 05:05:03.196130 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.196178 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 05:05:03.197434 8877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.197221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.195:50745)
0807 05:05:03.197313 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.197363 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 05:05:03.199659 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.199340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35215 (local address 127.1.17.194:39293)
0807 05:05:03.199534 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.199587 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 05:05:03.203004 8086 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:03.203161 8086 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:03.207190 8086 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64e07cde7c3f4ed295fcdac69c507145: Sending DeleteTablet for 3 replicas of tablet fcbabbaaa194494b8e76698d55282681
I20260807 05:05:03.207998 8695 tablet_service.cc:1514] Processing DeleteTablet for tablet fcbabbaaa194494b8e76698d55282681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:03 UTC) from {username='oryx', principal='oryx/127.1.17.253@KRBTEST.COM'} at 127.0.0.1:60156
I20260807 05:05:03.208010 8552 tablet_service.cc:1514] Processing DeleteTablet for tablet fcbabbaaa194494b8e76698d55282681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:03 UTC) from {username='oryx', principal='oryx/127.1.17.253@KRBTEST.COM'} at 127.0.0.1:58524
I20260807 05:05:03.208364 8895 tablet_replica.cc:330] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: stopping tablet replica
I20260807 05:05:03.208359 8400 tablet_service.cc:1514] Processing DeleteTablet for tablet fcbabbaaa194494b8e76698d55282681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:03 UTC) from {username='oryx', principal='oryx/127.1.17.253@KRBTEST.COM'} at 127.0.0.1:33708
I20260807 05:05:03.208604 8894 tablet_replica.cc:330] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: stopping tablet replica
I20260807 05:05:03.208701 8896 tablet_replica.cc:330] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135: stopping tablet replica
I20260807 05:05:03.208850 8896 raft_consensus.cc:2229] T fcbabbaaa194494b8e76698d55282681 P da9149d2f3864246beb77be7325bd135 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:05:03.209007 8895 raft_consensus.cc:2229] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:03.209049 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 8236
I20260807 05:05:03.208885 8894 raft_consensus.cc:2229] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:03.209312 8894 raft_consensus.cc:2258] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:03.209503 8895 raft_consensus.cc:2258] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:03.211069 8895 ts_tablet_manager.cc:1905] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:03.214707 8895 ts_tablet_manager.cc:1918] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:03.214818 8895 log.cc:1196] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/fcbabbaaa194494b8e76698d55282681
I20260807 05:05:03.215137 8895 ts_tablet_manager.cc:1939] T fcbabbaaa194494b8e76698d55282681 P 053d912d6fbf44cea636bdb2910c599a: Deleting consensus metadata
I20260807 05:05:03.215982 8071 catalog_manager.cc:4841] TS 053d912d6fbf44cea636bdb2910c599a (127.1.17.195:38633): tablet fcbabbaaa194494b8e76698d55282681 (table test-table [id=62703fac939641fcb069e604815d1349]) successfully deleted
I20260807 05:05:03.216297 8894 ts_tablet_manager.cc:1905] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:03.219915 8894 ts_tablet_manager.cc:1918] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:03.220024 8894 log.cc:1196] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/fcbabbaaa194494b8e76698d55282681
I20260807 05:05:03.220314 8894 ts_tablet_manager.cc:1939] T fcbabbaaa194494b8e76698d55282681 P 6696fca0c7dc4d709fd6621f51d20d0f: Deleting consensus metadata
I20260807 05:05:03.221179 8071 catalog_manager.cc:4841] TS 6696fca0c7dc4d709fd6621f51d20d0f (127.1.17.194:39727): tablet fcbabbaaa194494b8e76698d55282681 (table test-table [id=62703fac939641fcb069e604815d1349]) successfully deleted
I20260807 05:05:03.222932 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 8484
I20260807 05:05:03.223053 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:03.222692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:43031 (local address 127.0.0.1:33734)
0807 05:05:03.222847 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:03.222948 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:43031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 05:05:03.224700 8071 catalog_manager.cc:4584] TS da9149d2f3864246beb77be7325bd135 (127.1.17.193:43031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcbabbaaa194494b8e76698d55282681: Network error: Client connection negotiation failed: client connection to 127.1.17.193:43031: connect: Connection refused (error 111)
I20260807 05:05:03.230988 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 8628
I20260807 05:05:03.239580 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 7942
I20260807 05:05:03.246774 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 8053
2026-08-07T05:05:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5615 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:05:03 dist-test-slave-1x4g krb5kdc[8899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:05:03 dist-test-slave-1x4g krb5kdc[8899](info): set up 2 sockets
Aug 07 05:05:03 dist-test-slave-1x4g krb5kdc[8899](info): commencing operation
krb5kdc: starting...
W20260807 05:05:04.297912 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:05:04 dist-test-slave-1x4g krb5kdc[8899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:05:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:05:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab.
Entry for principal oryx/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab.
I20260807 05:05:04.428042 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:45337
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab
--principal=oryx/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:45337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:04.542658 8912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:04.542924 8912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:04.542964 8912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:04.546828 8912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:04.546880 8912 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 05:05:04.546897 8912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:04.546914 8912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:04.546931 8912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:04.546947 8912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:04.552060 8912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:45337
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.254.keytab
--principal=oryx/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:45337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:04.553037 8912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:04.553988 8912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:04.562271 8917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:04.562354 8918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:04.562352 8912 server_base.cc:1034] running on GCE node
W20260807 05:05:04.562291 8920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:04.563108 8912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:04.564282 8912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:04.565487 8912 hybrid_clock.cc:648] HybridClock initialized: now 1786079104565471 us; error 28 us; skew 500 ppm
Aug 07 05:05:04 dist-test-slave-1x4g krb5kdc[8899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:04.568441 8912 init.cc:377] Logged in from keytab as oryx/127.1.17.254@KRBTEST.COM (short username oryx)
I20260807 05:05:04.569380 8912 webserver.cc:460] Webserver started at http://127.1.17.254:40325/ using document root <none> and password file <none>
I20260807 05:05:04.569955 8912 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:04.570044 8912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:04.570266 8912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:04.572352 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "ffacf06f679f4275b7bf21d9568fb051"
format_stamp: "Formatted at 2026-08-07 05:05:04 on dist-test-slave-1x4g"
server_key: "cefd1bab08c82867d79ad94dd729871b"
server_key_iv: "896d591efb5cc6223607ad2cec66b042"
server_key_version: "encryptionkey@0"
I20260807 05:05:04.572991 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "ffacf06f679f4275b7bf21d9568fb051"
format_stamp: "Formatted at 2026-08-07 05:05:04 on dist-test-slave-1x4g"
server_key: "cefd1bab08c82867d79ad94dd729871b"
server_key_iv: "896d591efb5cc6223607ad2cec66b042"
server_key_version: "encryptionkey@0"
I20260807 05:05:04.576748 8912 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:05:04.579213 8927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:04.580377 8912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:04.580510 8912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "ffacf06f679f4275b7bf21d9568fb051"
format_stamp: "Formatted at 2026-08-07 05:05:04 on dist-test-slave-1x4g"
server_key: "cefd1bab08c82867d79ad94dd729871b"
server_key_iv: "896d591efb5cc6223607ad2cec66b042"
server_key_version: "encryptionkey@0"
I20260807 05:05:04.580616 8912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:04.603447 8912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:04.606441 8912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:04.606645 8912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:04.614691 8912 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:45337
I20260807 05:05:04.614735 8995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:45337 every 8 connection(s)
I20260807 05:05:04.615768 8912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:05:04.618549 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:04.624023 8996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Bootstrap starting.
I20260807 05:05:04.625059 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 8912
I20260807 05:05:04.625176 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:05:04.625434 1095 external_mini_cluster.cc:1428] Setting key e4d7318122e2024dfdb0f367fd03ad31
I20260807 05:05:04.626742 8996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:04.627509 8996 log.cc:824] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:04.629423 8996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: No bootstrap required, opened a new log
I20260807 05:05:04.632961 8996 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffacf06f679f4275b7bf21d9568fb051" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45337 } }
I20260807 05:05:04.633234 8996 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:04.633322 8996 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffacf06f679f4275b7bf21d9568fb051, State: Initialized, Role: FOLLOWER
I20260807 05:05:04.633781 8996 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffacf06f679f4275b7bf21d9568fb051" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45337 } }
I20260807 05:05:04.633929 8996 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:05:04.633999 8996 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:05:04.634104 8996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 05:05:04 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:04.635208 8996 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffacf06f679f4275b7bf21d9568fb051" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45337 } }
I20260807 05:05:04.635557 8996 leader_election.cc:302] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffacf06f679f4275b7bf21d9568fb051; no voters:
I20260807 05:05:04.635906 8996 leader_election.cc:288] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:05:04.636021 9001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:04.636241 9001 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [term 1 LEADER]: Becoming Leader. State: Replica: ffacf06f679f4275b7bf21d9568fb051, State: Running, Role: LEADER
I20260807 05:05:04.636656 9001 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffacf06f679f4275b7bf21d9568fb051" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45337 } }
I20260807 05:05:04.637282 8996 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:04.637856 9001 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffacf06f679f4275b7bf21d9568fb051. Latest consensus state: current_term: 1 leader_uuid: "ffacf06f679f4275b7bf21d9568fb051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffacf06f679f4275b7bf21d9568fb051" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45337 } } }
I20260807 05:05:04.637988 9001 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:04.638337 9002 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffacf06f679f4275b7bf21d9568fb051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffacf06f679f4275b7bf21d9568fb051" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45337 } } }
I20260807 05:05:04.638456 9002 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:04.641458 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:04.626553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49344 (local address 127.1.17.254:45337)
0807 05:05:04.627079 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:04.627095 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:04.627130 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:05:04.627807 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:04.627839 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:04.628248 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:04.628589 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:04.629432 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.630450 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.631295 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.631788 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.631961 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:04.634799 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:04.634804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:04.634817 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:04.634842 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:04.636955 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:04.637493 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:04.637499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:04.637505 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:04.637667 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:04.638842 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:04.638845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:04.638847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:04.639421 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:04.639661 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:04.640201 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 05:05:04.640493 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":160,"threads_started":1}
W20260807 05:05:04.642936 9018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:05:04.643018 9018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:05:04.643113 9006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:05:04.643994 9006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:05:04.649657 9006 catalog_manager.cc:1347] Generated new cluster ID: 3af7bf7aaa1343d493d16ed72a4f5af4
I20260807 05:05:04.649724 9006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:05:04.666582 9006 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:05:04.667649 9006 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:05:04.677753 9006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Generated new TSK 0
I20260807 05:05:04.678471 9006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab.
Entry for principal oryx/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab.
I20260807 05:05:04.717444 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45337
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab
--principal=oryx/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:04.837273 9024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:04.837587 9024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:04.837633 9024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:04.841861 9024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:04.841920 9024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:04.841964 9024 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:05:04.842003 9024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:05:04.847265 9024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab
--principal=oryx/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:04.848333 9024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:04.849391 9024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:04.859022 9029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:04.859022 9033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:04.859223 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:04.860028 9024 server_base.cc:1034] running on GCE node
I20260807 05:05:04.860445 9024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:04.861140 9024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:04.862313 9024 hybrid_clock.cc:648] HybridClock initialized: now 1786079104862281 us; error 41 us; skew 500 ppm
Aug 07 05:05:04 dist-test-slave-1x4g krb5kdc[8899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:04.865489 9024 init.cc:377] Logged in from keytab as oryx/127.1.17.193@KRBTEST.COM (short username oryx)
I20260807 05:05:04.866431 9024 webserver.cc:460] Webserver started at http://127.1.17.193:39483/ using document root <none> and password file <none>
I20260807 05:05:04.867039 9024 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:04.867132 9024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:04.867385 9024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:04.869716 9024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "d5c349eaf853433094b00eebbf7ed7cf"
format_stamp: "Formatted at 2026-08-07 05:05:04 on dist-test-slave-1x4g"
server_key: "37e38b7b3c3d679b30a252e7241ae183"
server_key_iv: "2d09e3992d6ac429a42f87730b4f93b3"
server_key_version: "encryptionkey@0"
I20260807 05:05:04.870314 9024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "d5c349eaf853433094b00eebbf7ed7cf"
format_stamp: "Formatted at 2026-08-07 05:05:04 on dist-test-slave-1x4g"
server_key: "37e38b7b3c3d679b30a252e7241ae183"
server_key_iv: "2d09e3992d6ac429a42f87730b4f93b3"
server_key_version: "encryptionkey@0"
I20260807 05:05:04.874564 9024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:05:04.877230 9041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:04.878489 9024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:04.878641 9024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "d5c349eaf853433094b00eebbf7ed7cf"
format_stamp: "Formatted at 2026-08-07 05:05:04 on dist-test-slave-1x4g"
server_key: "37e38b7b3c3d679b30a252e7241ae183"
server_key_iv: "2d09e3992d6ac429a42f87730b4f93b3"
server_key_version: "encryptionkey@0"
I20260807 05:05:04.878763 9024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:04.906032 9024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:04.909866 9024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:04.910116 9024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:04.911911 9024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:04.911989 9024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:04.912070 9024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:04.912128 9024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:05:04 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:04.924639 9024 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:39721
I20260807 05:05:04.924714 9163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:39721 every 8 connection(s)
I20260807 05:05:04.925697 9024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:05:04.927002 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:04.912724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:37365 (local address 127.1.17.254:45337)
0807 05:05:04.912880 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:04.912884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:04.914005 (+ 1121us) server_negotiation.cc:407] Connection header received
0807 05:05:04.915050 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:04.915054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:04.915114 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:04.915221 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:04.916839 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.917521 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.918753 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.919000 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.919110 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:04.923029 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:04.923032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:04.923034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:04.923061 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:04.924697 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:04.925262 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:04.925266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:04.925268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:04.925358 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:04.925700 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:04.925703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:04.925705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:04.925981 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:04.926090 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:04.926691 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 05:05:04.926824 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 05:05:04.927759 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:04.913334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45337 (local address 127.1.17.193:37365)
0807 05:05:04.913847 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:04.913892 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:04.914835 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:04.915426 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:04.915435 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:04.915933 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:04.916655 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:04.916671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.917644 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.917648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:04.918627 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:04.918637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.919157 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.919162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:04.919333 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:04.919955 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:04.919985 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:04.922846 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:04.924857 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:04.924863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:04.924879 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:04.925152 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:04.925450 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:04.925453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:04.925455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:04.925599 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:04.926090 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:04.926098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:04.926428 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 05:05:04.926628 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 05:05:04.926953 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":140,"threads_started":1}
I20260807 05:05:04.929263 9164 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45337
I20260807 05:05:04.929488 9164 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:04.930186 9164 heartbeater.cc:511] Master 127.1.17.254:45337 requested a full tablet report, sending...
I20260807 05:05:04.933326 8943 ts_manager.cc:194] Registered new tserver with Master: d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721)
I20260807 05:05:04.934973 8943 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:37365
I20260807 05:05:04.935324 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 9024
I20260807 05:05:04.935475 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:05:04.935770 1095 external_mini_cluster.cc:1428] Setting key 1dc9a15116174db11a8878cd0e30cba9
I20260807 05:05:04.942309 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:04.931905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:56783 (local address 127.1.17.254:45337)
0807 05:05:04.932032 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:04.932036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:04.932258 (+ 222us) server_negotiation.cc:407] Connection header received
0807 05:05:04.932410 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:04.932413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:04.932463 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:04.932531 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:04.933155 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.933725 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.934963 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.935258 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.935366 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:04.936801 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:04.936807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:04.936810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:04.936837 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:04.940587 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:04.941103 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:04.941105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:04.941107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:04.941189 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:04.941509 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:04.941511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:04.941512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:04.941684 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:04.941767 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:04.942082 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 05:05:04.942190 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:05:04.942536 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:04.931851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45337 (local address 127.1.17.193:56783)
0807 05:05:04.932167 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:04.932184 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:04.932297 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:04.932577 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:04.932580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:04.932884 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:04.933036 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:04.933045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.933864 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.933868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:04.934795 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:04.934808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:04.935406 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:04.935411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:04.935515 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:04.936107 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:04.936124 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:04.936587 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:04.940687 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:04.940693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:04.940699 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:04.941002 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:04.941284 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:04.941290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:04.941295 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:04.941407 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:04.941780 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:04.941786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:04.941902 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 05:05:04.942022 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 05:05:04.942219 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for oryx/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab.
Entry for principal oryx/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab.
I20260807 05:05:04.965595 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45337
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab
--principal=oryx/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:05.081221 9174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:05.081506 9174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:05.081580 9174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:05.085901 9174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:05.085966 9174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:05.086023 9174 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:05:05.086107 9174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:05:05.091384 9174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab
--principal=oryx/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:05.092465 9174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:05.093536 9174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:05.103629 9180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:05.103624 9182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:05.103623 9179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:05.103739 9174 server_base.cc:1034] running on GCE node
I20260807 05:05:05.104368 9174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:05.105088 9174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:05.106256 9174 hybrid_clock.cc:648] HybridClock initialized: now 1786079105106237 us; error 37 us; skew 500 ppm
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079105, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:05.109270 9174 init.cc:377] Logged in from keytab as oryx/127.1.17.194@KRBTEST.COM (short username oryx)
I20260807 05:05:05.110205 9174 webserver.cc:460] Webserver started at http://127.1.17.194:42085/ using document root <none> and password file <none>
I20260807 05:05:05.110781 9174 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:05.110870 9174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:05.111104 9174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:05.113235 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "ffd22e5d5fb443aab8dc294242ee72e5"
format_stamp: "Formatted at 2026-08-07 05:05:05 on dist-test-slave-1x4g"
server_key: "d9a819dd097777a219ed249737d2a190"
server_key_iv: "edffabe12fe904dbf8dc04a7d77c1f5b"
server_key_version: "encryptionkey@0"
I20260807 05:05:05.113798 9174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "ffd22e5d5fb443aab8dc294242ee72e5"
format_stamp: "Formatted at 2026-08-07 05:05:05 on dist-test-slave-1x4g"
server_key: "d9a819dd097777a219ed249737d2a190"
server_key_iv: "edffabe12fe904dbf8dc04a7d77c1f5b"
server_key_version: "encryptionkey@0"
I20260807 05:05:05.117724 9174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:05:05.120131 9189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:05.121433 9174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:05:05.121582 9174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "ffd22e5d5fb443aab8dc294242ee72e5"
format_stamp: "Formatted at 2026-08-07 05:05:05 on dist-test-slave-1x4g"
server_key: "d9a819dd097777a219ed249737d2a190"
server_key_iv: "edffabe12fe904dbf8dc04a7d77c1f5b"
server_key_version: "encryptionkey@0"
I20260807 05:05:05.121695 9174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:05.142803 9174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:05.145900 9174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:05.146114 9174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:05.147671 9174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:05.147740 9174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:05.147809 9174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:05.147854 9174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079105, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:05.160202 9174 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:37761
I20260807 05:05:05.160255 9308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:37761 every 8 connection(s)
I20260807 05:05:05.161320 9174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:05:05.162598 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.148513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:47439 (local address 127.1.17.254:45337)
0807 05:05:05.148642 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.148646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.149847 (+ 1201us) server_negotiation.cc:407] Connection header received
0807 05:05:05.151042 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.151047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.151119 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.151227 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:05.152769 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.153430 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.154694 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.154993 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.155106 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.158415 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:05.158419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:05.158421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:05.158448 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:05.160241 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.160838 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.160842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.160844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.160946 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.161312 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.161315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.161317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.161572 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:05.161677 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.162303 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.162434 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:05:05.162969 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 9174
I20260807 05:05:05.163111 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:05:05.163405 1095 external_mini_cluster.cc:1428] Setting key f38233f7235d5d8833c70ebd1df88bba
I20260807 05:05:05.163563 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.149174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45337 (local address 127.1.17.194:47439)
0807 05:05:05.149680 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.149727 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.150719 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.151528 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.151536 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.151989 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:05.152595 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.152618 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.153572 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.153576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.154554 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.154564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.155155 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.155159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.155334 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.156053 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:05.156077 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:05.158226 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:05.160415 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.160421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.160438 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.160706 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.161051 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.161054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.161056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.161205 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.161678 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:05.161687 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:05.162017 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.162239 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.162596 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":131,"threads_started":1}
I20260807 05:05:05.164906 9309 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45337
I20260807 05:05:05.165100 9309 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:05.165609 9309 heartbeater.cc:511] Master 127.1.17.254:45337 requested a full tablet report, sending...
I20260807 05:05:05.166669 8943 ts_manager.cc:194] Registered new tserver with Master: ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761)
I20260807 05:05:05.167346 8943 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:47439
WARNING: no policy specified for oryx/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:05:05.175706 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.167897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:51515 (local address 127.1.17.254:45337)
0807 05:05:05.168027 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.168031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.168917 (+ 886us) server_negotiation.cc:407] Connection header received
0807 05:05:05.169087 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.169095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.169134 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.169236 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:05.169689 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.170220 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.171043 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.171262 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.171368 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.172520 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:05.172529 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:05.172530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:05.172552 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:05.173962 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.174400 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.174409 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.174410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.174489 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.174765 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.174769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.174770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.174997 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:05.175110 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.175421 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.175538 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:05:05.175714 9316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.167597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45337 (local address 127.1.17.194:51515)
0807 05:05:05.167919 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.168844 (+ 925us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.168980 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.169256 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.169259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.169471 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:05.169571 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.169577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.170343 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.170347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.170921 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.170929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.171375 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.171378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.171473 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.172040 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:05.172052 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:05.172384 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:05.174081 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.174084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.174087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.174293 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.174581 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.174584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.174586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.174675 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.175100 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:05.175103 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:05.175204 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.175352 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.175501 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab.
Entry for principal oryx/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab.
I20260807 05:05:05.194031 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45337
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab
--principal=oryx/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:05.310619 9318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:05.310879 9318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:05.310953 9318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:05.314957 9318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:05.315016 9318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:05.315076 9318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 05:05:05.315143 9318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:05:05.320425 9318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab
--principal=oryx/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.17.254:45337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:05.321475 9318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:05.322477 9318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:05:05.332355 9318 server_base.cc:1034] running on GCE node
W20260807 05:05:05.332304 9323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:05.332288 9326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:05.332288 9324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:05.333022 9318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:05.333652 9318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:05.334833 9318 hybrid_clock.cc:648] HybridClock initialized: now 1786079105334820 us; error 30 us; skew 500 ppm
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079105, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:05.337635 9318 init.cc:377] Logged in from keytab as oryx/127.1.17.195@KRBTEST.COM (short username oryx)
I20260807 05:05:05.338598 9318 webserver.cc:460] Webserver started at http://127.1.17.195:35401/ using document root <none> and password file <none>
I20260807 05:05:05.339184 9318 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:05.339274 9318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:05.339499 9318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:05.341606 9318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "978b820670db4a679970cbb3b9a2e13c"
format_stamp: "Formatted at 2026-08-07 05:05:05 on dist-test-slave-1x4g"
server_key: "8cec20c0aab9c25c6666b1ee30fc0642"
server_key_iv: "7953f3b8bb0bb807e8527cecf15407ff"
server_key_version: "encryptionkey@0"
I20260807 05:05:05.342168 9318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "978b820670db4a679970cbb3b9a2e13c"
format_stamp: "Formatted at 2026-08-07 05:05:05 on dist-test-slave-1x4g"
server_key: "8cec20c0aab9c25c6666b1ee30fc0642"
server_key_iv: "7953f3b8bb0bb807e8527cecf15407ff"
server_key_version: "encryptionkey@0"
I20260807 05:05:05.346056 9318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:05:05.348466 9333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:05.349742 9318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:05.349874 9318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "978b820670db4a679970cbb3b9a2e13c"
format_stamp: "Formatted at 2026-08-07 05:05:05 on dist-test-slave-1x4g"
server_key: "8cec20c0aab9c25c6666b1ee30fc0642"
server_key_iv: "7953f3b8bb0bb807e8527cecf15407ff"
server_key_version: "encryptionkey@0"
I20260807 05:05:05.350003 9318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:05.370640 9318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:05.373608 9318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:05.373801 9318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:05.375303 9318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:05.375371 9318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:05.375434 9318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:05.375474 9318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079105, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:05.388007 9455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:41019 every 8 connection(s)
I20260807 05:05:05.388031 9318 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:41019
I20260807 05:05:05.389269 9318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:05:05.390522 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.376061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:37585 (local address 127.1.17.254:45337)
0807 05:05:05.376230 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.376237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.377155 (+ 918us) server_negotiation.cc:407] Connection header received
0807 05:05:05.378152 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.378156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.378215 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.378298 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:05.379559 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.380216 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.381861 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.382091 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.382214 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.386015 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:05.386024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:05.386027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:05.386057 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:05.388056 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.388712 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.388715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.388716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.388849 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.389272 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.389276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.389278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.389529 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:05.389642 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.390279 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.390400 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:05:05.391305 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 9318
I20260807 05:05:05.391443 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:05:05.391489 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.376491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45337 (local address 127.1.17.195:37585)
0807 05:05:05.376988 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.377040 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.377928 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.378606 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.378613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.378933 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:05.379399 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.379423 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.380424 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.380429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.381725 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.381736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.382255 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.382259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.382498 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.383167 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:05.383189 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:05.385745 (+ 2556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:05.388210 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.388216 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.388233 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.388592 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.389003 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.389007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.389009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.389162 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.389647 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:05.389656 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:05.389992 (+ 336us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.390219 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.390542 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":143,"threads_started":1}
I20260807 05:05:05.391752 1095 external_mini_cluster.cc:1428] Setting key a6c60aea8093e8764c4c9bc41ad62c68
I20260807 05:05:05.392869 9456 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45337
I20260807 05:05:05.393025 9456 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:05.393575 9456 heartbeater.cc:511] Master 127.1.17.254:45337 requested a full tablet report, sending...
I20260807 05:05:05.394562 8942 ts_manager.cc:194] Registered new tserver with Master: 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:05.395304 8942 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:37585
I20260807 05:05:05.401849 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.395039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:57937 (local address 127.1.17.254:45337)
0807 05:05:05.395162 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.395166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.395309 (+ 143us) server_negotiation.cc:407] Connection header received
0807 05:05:05.395543 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.395546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.395597 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.395684 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:05.396115 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.396631 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.397505 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.397728 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.397833 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.398919 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:05.398923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:05.398925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:05.398954 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:05.400200 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.400643 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.400647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.400649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.400731 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.401037 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.401041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.401042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.401218 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:05.401323 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.401585 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.401698 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:05:05.401860 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.394926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45337 (local address 127.1.17.195:57937)
0807 05:05:05.395215 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.395231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.395335 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.395703 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.395706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.395909 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:05.396003 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.396009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.396754 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.396757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.397373 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.397382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.397839 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.397843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.397936 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.398448 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:05.398460 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:05.398772 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:05.400324 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.400327 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.400329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.400542 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.400843 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.400848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.400850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.400939 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.401313 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:05.401316 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:05.401422 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.401529 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.401678 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
I20260807 05:05:05.405637 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079105, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:05:05.431463 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:05.425011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.1.17.254:45337)
0807 05:05:05.425145 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.425149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.425161 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:05:05.425206 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.425208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.425247 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.425320 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:05.426157 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.426689 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.427551 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.427771 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.427850 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.431283 (+ 3433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49360: BlockingRecv error: recv got EOF from 127.0.0.1:49360 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079105, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:05.440860 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.432942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.1.17.254:45337)
0807 05:05:05.433076 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.433080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.433206 (+ 126us) server_negotiation.cc:407] Connection header received
0807 05:05:05.433307 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.433310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.433350 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.433425 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:05.433967 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.434469 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.435209 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.435408 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.435490 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.437853 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:05.437856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:05.437858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:05.437881 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:05.439160 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.439575 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.439578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.439580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.439668 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.439940 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.439944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.439946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.440137 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:05.440278 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.440508 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.440639 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:05:05.444613 8942 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:05:05.449419 8942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:05.464870 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.458759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.1.17.194:37761)
0807 05:05:05.459126 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.459143 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.459924 (+ 781us) server_negotiation.cc:407] Connection header received
0807 05:05:05.460014 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.460023 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.460256 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.460422 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.460739 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.462018 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.463458 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.464060 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.464188 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.464374 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.464590 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.464688 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":152,"threads_started":1}
I20260807 05:05:05.464902 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.458590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:37761 (local address 127.0.0.1:44268)
0807 05:05:05.459553 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.459622 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.459772 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.460408 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.460415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.460438 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.460595 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.460606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.462121 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.462126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.463337 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.463347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.464189 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.464193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.464341 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.464394 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.464558 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.464739 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":157,"thread_start_us":77,"threads_started":1}
I20260807 05:05:05.465154 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.458834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41019 (local address 127.0.0.1:49224)
0807 05:05:05.459819 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.459834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.459937 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.460683 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.460687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.460707 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.460851 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.460859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.462177 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.462180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.463380 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.463390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.464700 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.464703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.464842 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.464866 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.464935 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.464998 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"mutex_wait_us":37,"thread_start_us":82,"threads_started":1}
I20260807 05:05:05.465727 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.459413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49224 (local address 127.1.17.195:41019)
0807 05:05:05.460208 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.460219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.460244 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:05:05.460323 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.460328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.460525 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.460689 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.460968 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.462048 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.463538 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.464351 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.464460 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.464635 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.465465 (+ 830us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.465574 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":129,"threads_started":1}
I20260807 05:05:05.466177 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.459335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39721 (local address 127.0.0.1:37764)
0807 05:05:05.460157 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.460170 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.460253 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.460967 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.460970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.460985 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.461059 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.461063 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.462022 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.462026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.464341 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.464351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.465758 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.465763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.465880 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.465904 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.465978 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.466049 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":932,"thread_start_us":82,"threads_started":1}
I20260807 05:05:05.466666 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.459531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.1.17.193:39721)
0807 05:05:05.459891 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.459898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.460359 (+ 461us) server_negotiation.cc:407] Connection header received
0807 05:05:05.460444 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.460449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.460661 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.460821 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.461177 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.461905 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.464491 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.465428 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.465540 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.465685 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.466399 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.466516 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":85,"threads_started":1}
I20260807 05:05:05.468529 9239 tablet_service.cc:1467] Processing CreateTablet for tablet d42805eda5a74653a7853d24d904a329 (DEFAULT_TABLE table=test-table [id=3f2169212fbf4bcc9b6ef8d8c48b7aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:05.468641 9385 tablet_service.cc:1467] Processing CreateTablet for tablet d42805eda5a74653a7853d24d904a329 (DEFAULT_TABLE table=test-table [id=3f2169212fbf4bcc9b6ef8d8c48b7aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:05.469130 9093 tablet_service.cc:1467] Processing CreateTablet for tablet d42805eda5a74653a7853d24d904a329 (DEFAULT_TABLE table=test-table [id=3f2169212fbf4bcc9b6ef8d8c48b7aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:05.469619 9239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42805eda5a74653a7853d24d904a329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.469744 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42805eda5a74653a7853d24d904a329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.470038 9093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d42805eda5a74653a7853d24d904a329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.476902 9488 tablet_bootstrap.cc:492] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Bootstrap starting.
I20260807 05:05:05.477738 9490 tablet_bootstrap.cc:492] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Bootstrap starting.
I20260807 05:05:05.479285 9488 tablet_bootstrap.cc:654] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.479642 9490 tablet_bootstrap.cc:654] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.480189 9488 log.cc:824] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:05.480400 9490 log.cc:824] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:05.480996 9491 tablet_bootstrap.cc:492] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Bootstrap starting.
I20260807 05:05:05.482455 9488 tablet_bootstrap.cc:492] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: No bootstrap required, opened a new log
I20260807 05:05:05.482657 9490 tablet_bootstrap.cc:492] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: No bootstrap required, opened a new log
I20260807 05:05:05.482765 9488 ts_tablet_manager.cc:1397] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:05:05.482825 9490 ts_tablet_manager.cc:1397] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:05:05.482955 9491 tablet_bootstrap.cc:654] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.483601 9491 log.cc:824] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:05.485546 9491 tablet_bootstrap.cc:492] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: No bootstrap required, opened a new log
I20260807 05:05:05.485749 9491 ts_tablet_manager.cc:1397] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:05:05.487306 9490 raft_consensus.cc:348] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.487306 9488 raft_consensus.cc:348] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.487586 9490 raft_consensus.cc:374] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.487586 9488 raft_consensus.cc:374] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.487643 9488 raft_consensus.cc:729] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd22e5d5fb443aab8dc294242ee72e5, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.487653 9490 raft_consensus.cc:729] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c349eaf853433094b00eebbf7ed7cf, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.488121 9490 consensus_queue.cc:260] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.488121 9488 consensus_queue.cc:260] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.488845 9164 heartbeater.cc:503] Master 127.1.17.254:45337 was elected leader, sending a full tablet report...
I20260807 05:05:05.489198 9488 ts_tablet_manager.cc:1428] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:05:05.489240 9309 heartbeater.cc:503] Master 127.1.17.254:45337 was elected leader, sending a full tablet report...
I20260807 05:05:05.489444 9491 raft_consensus.cc:348] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.489704 9491 raft_consensus.cc:374] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.489764 9491 raft_consensus.cc:729] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978b820670db4a679970cbb3b9a2e13c, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.490224 9491 consensus_queue.cc:260] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.490520 9490 ts_tablet_manager.cc:1428] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 05:05:05.491008 9456 heartbeater.cc:503] Master 127.1.17.254:45337 was elected leader, sending a full tablet report...
I20260807 05:05:05.491245 9491 ts_tablet_manager.cc:1428] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:05:05.525491 9498 raft_consensus.cc:482] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:05.525727 9498 raft_consensus.cc:504] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.526885 9498 leader_election.cc:288] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761), d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721)
I20260807 05:05:05.531026 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.527172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:37761 (local address 127.1.17.195:42839)
0807 05:05:05.527304 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.527334 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.527433 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.527683 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.527686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.527705 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.527829 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.527837 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.528861 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.528864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.530028 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.530039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.530646 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.530650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.530765 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.530795 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.530858 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.530917 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260807 05:05:05.531117 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.527278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:42839 (local address 127.1.17.194:37761)
0807 05:05:05.527414 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.527419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.527435 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:05:05.527535 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.527541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.527592 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.527672 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.527950 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.528735 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.530145 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.530536 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.530638 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.530680 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.530892 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.530972 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:05:05.531842 9262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42805eda5a74653a7853d24d904a329" candidate_uuid: "978b820670db4a679970cbb3b9a2e13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" is_pre_election: true
I20260807 05:05:05.531929 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.527500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39721 (local address 127.1.17.195:37053)
0807 05:05:05.527918 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.527933 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.528019 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.528270 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.528273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.528297 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.528420 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.528426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.529318 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.529321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.530679 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.530688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.531522 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.531525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.531594 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.531611 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.531708 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.531771 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":20,"thread_start_us":139,"threads_started":1}
I20260807 05:05:05.531929 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.527615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:37053 (local address 127.1.17.193:39721)
0807 05:05:05.527768 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.527773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.528007 (+ 234us) server_negotiation.cc:407] Connection header received
0807 05:05:05.528095 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.528098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.528152 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.528248 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.528592 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.529213 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.530800 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.531399 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.531523 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.531561 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.531728 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.531808 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 05:05:05.532195 9262 raft_consensus.cc:2454] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978b820670db4a679970cbb3b9a2e13c in term 0.
I20260807 05:05:05.532516 9115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42805eda5a74653a7853d24d904a329" candidate_uuid: "978b820670db4a679970cbb3b9a2e13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c349eaf853433094b00eebbf7ed7cf" is_pre_election: true
I20260807 05:05:05.532727 9336 leader_election.cc:302] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978b820670db4a679970cbb3b9a2e13c, ffd22e5d5fb443aab8dc294242ee72e5; no voters:
I20260807 05:05:05.532866 9115 raft_consensus.cc:2454] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978b820670db4a679970cbb3b9a2e13c in term 0.
I20260807 05:05:05.533066 9498 raft_consensus.cc:2781] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:05.533166 9498 raft_consensus.cc:482] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:05.533229 9498 raft_consensus.cc:3037] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.534322 9498 raft_consensus.cc:504] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.534680 9498 leader_election.cc:288] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [CANDIDATE]: Term 1 election: Requested vote from peers ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761), d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721)
I20260807 05:05:05.535069 9115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42805eda5a74653a7853d24d904a329" candidate_uuid: "978b820670db4a679970cbb3b9a2e13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c349eaf853433094b00eebbf7ed7cf"
I20260807 05:05:05.535099 9262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d42805eda5a74653a7853d24d904a329" candidate_uuid: "978b820670db4a679970cbb3b9a2e13c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd22e5d5fb443aab8dc294242ee72e5"
I20260807 05:05:05.535231 9115 raft_consensus.cc:3037] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.535229 9262 raft_consensus.cc:3037] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.536267 9115 raft_consensus.cc:2454] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978b820670db4a679970cbb3b9a2e13c in term 1.
I20260807 05:05:05.536583 9336 leader_election.cc:302] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978b820670db4a679970cbb3b9a2e13c, d5c349eaf853433094b00eebbf7ed7cf; no voters:
I20260807 05:05:05.536762 9498 raft_consensus.cc:2781] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:05.536804 9262 raft_consensus.cc:2454] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978b820670db4a679970cbb3b9a2e13c in term 1.
I20260807 05:05:05.537060 9498 raft_consensus.cc:686] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 1 LEADER]: Becoming Leader. State: Replica: 978b820670db4a679970cbb3b9a2e13c, State: Running, Role: LEADER
I20260807 05:05:05.537635 9498 consensus_queue.cc:237] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.541098 8942 catalog_manager.cc:5495] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c reported cstate change: term changed from 0 to 1, leader changed from <none> to 978b820670db4a679970cbb3b9a2e13c (127.1.17.195). New cstate: current_term: 1 leader_uuid: "978b820670db4a679970cbb3b9a2e13c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } health_report { overall_health: HEALTHY } } }
I20260807 05:05:05.564462 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.560986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.1.17.195:41019)
0807 05:05:05.561129 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.561133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.561146 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:05.561302 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.561304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.561359 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.561449 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:05.561670 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.562179 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.563036 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.563250 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.563325 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.563581 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.564018 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.564115 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.564237 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.564287 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:05:05.570345 9115 raft_consensus.cc:1261] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Refusing update from remote peer 978b820670db4a679970cbb3b9a2e13c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:05.570401 9262 raft_consensus.cc:1261] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Refusing update from remote peer 978b820670db4a679970cbb3b9a2e13c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:05.571121 9498 consensus_queue.cc:1035] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:05.571359 9500 consensus_queue.cc:1035] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:05.578348 9503 mvcc.cc:205] Tried to move back new op lower bound from 7315780016408924160 to 7315780016285204480. Current Snapshot: MvccSnapshot[applied={T|T < 7315780016408924160}]
I20260807 05:05:05.580245 9504 mvcc.cc:205] Tried to move back new op lower bound from 7315780016408924160 to 7315780016285204480. Current Snapshot: MvccSnapshot[applied={T|T < 7315780016408924160}]
I20260807 05:05:05.580932 9505 mvcc.cc:205] Tried to move back new op lower bound from 7315780016408924160 to 7315780016285204480. Current Snapshot: MvccSnapshot[applied={T|T < 7315780016408924160 or (T in {7315780016408924160})}]
I20260807 05:05:05.588013 8942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:05.588189 8942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:05.590538 8942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Sending DeleteTablet for 3 replicas of tablet d42805eda5a74653a7853d24d904a329
I20260807 05:05:05.591352 9093 tablet_service.cc:1514] Processing DeleteTablet for tablet d42805eda5a74653a7853d24d904a329 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:05 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:37764
I20260807 05:05:05.591357 9239 tablet_service.cc:1514] Processing DeleteTablet for tablet d42805eda5a74653a7853d24d904a329 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:05 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:44268
I20260807 05:05:05.591363 9385 tablet_service.cc:1514] Processing DeleteTablet for tablet d42805eda5a74653a7853d24d904a329 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:05 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:49224
I20260807 05:05:05.591986 9513 tablet_replica.cc:330] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: stopping tablet replica
I20260807 05:05:05.592294 9513 raft_consensus.cc:2229] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:05:05.592679 8942 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:05:05.592893 9515 tablet_replica.cc:330] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: stopping tablet replica
I20260807 05:05:05.593199 9515 raft_consensus.cc:2229] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 05:05:05.593327 8942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:05.593499 9515 raft_consensus.cc:2258] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:05.593797 9513 raft_consensus.cc:2258] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:05.594280 9514 tablet_replica.cc:330] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: stopping tablet replica
I20260807 05:05:05.594556 9514 raft_consensus.cc:2229] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:05.594830 9514 raft_consensus.cc:2258] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:05.596166 9513 ts_tablet_manager.cc:1905] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:05.598158 9093 tablet_service.cc:1467] Processing CreateTablet for tablet 4627a169ec274e5a815b7e832ffd5a48 (DEFAULT_TABLE table=test-table [id=89ee201cf95a4952b67c373ba50e7353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:05.598356 9385 tablet_service.cc:1467] Processing CreateTablet for tablet 4627a169ec274e5a815b7e832ffd5a48 (DEFAULT_TABLE table=test-table [id=89ee201cf95a4952b67c373ba50e7353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:05.598480 9093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4627a169ec274e5a815b7e832ffd5a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.598608 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4627a169ec274e5a815b7e832ffd5a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.600442 9490 tablet_bootstrap.cc:492] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Bootstrap starting.
I20260807 05:05:05.600855 9491 tablet_bootstrap.cc:492] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Bootstrap starting.
I20260807 05:05:05.601464 9239 tablet_service.cc:1467] Processing CreateTablet for tablet 4627a169ec274e5a815b7e832ffd5a48 (DEFAULT_TABLE table=test-table [id=89ee201cf95a4952b67c373ba50e7353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:05.601589 9490 tablet_bootstrap.cc:654] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.601751 9239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4627a169ec274e5a815b7e832ffd5a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.601781 9491 tablet_bootstrap.cc:654] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.602892 9490 tablet_bootstrap.cc:492] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: No bootstrap required, opened a new log
I20260807 05:05:05.602977 9490 ts_tablet_manager.cc:1397] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:05:05.603338 9514 ts_tablet_manager.cc:1905] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:05.603483 9490 raft_consensus.cc:348] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.603590 9490 raft_consensus.cc:374] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.603621 9490 raft_consensus.cc:729] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c349eaf853433094b00eebbf7ed7cf, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.603747 9490 consensus_queue.cc:260] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.604022 9490 ts_tablet_manager.cc:1428] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:05.604182 9488 tablet_bootstrap.cc:492] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Bootstrap starting.
I20260807 05:05:05.605172 9488 tablet_bootstrap.cc:654] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.605180 9515 ts_tablet_manager.cc:1905] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:05.606503 9514 ts_tablet_manager.cc:1918] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:05.606614 9514 log.cc:1196] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/d42805eda5a74653a7853d24d904a329
I20260807 05:05:05.606971 9514 ts_tablet_manager.cc:1939] T d42805eda5a74653a7853d24d904a329 P d5c349eaf853433094b00eebbf7ed7cf: Deleting consensus metadata
I20260807 05:05:05.607069 9488 tablet_bootstrap.cc:492] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: No bootstrap required, opened a new log
I20260807 05:05:05.607172 9488 ts_tablet_manager.cc:1397] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:05:05.607549 9488 raft_consensus.cc:348] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.607659 9488 raft_consensus.cc:374] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.607699 9488 raft_consensus.cc:729] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd22e5d5fb443aab8dc294242ee72e5, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.607717 9515 ts_tablet_manager.cc:1918] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:05.607813 9515 log.cc:1196] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/d42805eda5a74653a7853d24d904a329
I20260807 05:05:05.607838 9488 consensus_queue.cc:260] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.608125 9515 ts_tablet_manager.cc:1939] T d42805eda5a74653a7853d24d904a329 P ffd22e5d5fb443aab8dc294242ee72e5: Deleting consensus metadata
I20260807 05:05:05.608330 8929 catalog_manager.cc:4841] TS d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721): tablet d42805eda5a74653a7853d24d904a329 (table test-table [id=3f2169212fbf4bcc9b6ef8d8c48b7aba]) successfully deleted
I20260807 05:05:05.608129 9488 ts_tablet_manager.cc:1428] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:05.609391 8929 catalog_manager.cc:4841] TS ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761): tablet d42805eda5a74653a7853d24d904a329 (table test-table [id=3f2169212fbf4bcc9b6ef8d8c48b7aba]) successfully deleted
I20260807 05:05:05.609519 9513 ts_tablet_manager.cc:1918] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:05.609616 9513 log.cc:1196] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/d42805eda5a74653a7853d24d904a329
I20260807 05:05:05.609912 9513 ts_tablet_manager.cc:1939] T d42805eda5a74653a7853d24d904a329 P 978b820670db4a679970cbb3b9a2e13c: Deleting consensus metadata
I20260807 05:05:05.610931 8931 catalog_manager.cc:4841] TS 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019): tablet d42805eda5a74653a7853d24d904a329 (table test-table [id=3f2169212fbf4bcc9b6ef8d8c48b7aba]) successfully deleted
I20260807 05:05:05.611788 9491 tablet_bootstrap.cc:492] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: No bootstrap required, opened a new log
I20260807 05:05:05.611907 9491 ts_tablet_manager.cc:1397] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260807 05:05:05.612269 9491 raft_consensus.cc:348] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.612385 9491 raft_consensus.cc:374] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.612461 9491 raft_consensus.cc:729] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978b820670db4a679970cbb3b9a2e13c, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.612600 9491 consensus_queue.cc:260] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.612928 9491 ts_tablet_manager.cc:1428] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260807 05:05:05.640261 9457 tablet.cc:2369] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:05:05.659663 9497 raft_consensus.cc:482] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:05.659835 9497 raft_consensus.cc:504] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.661216 9497 leader_election.cc:288] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
W20260807 05:05:05.662567 9310 tablet.cc:2369] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:05:05.665879 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.661434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39721 (local address 127.1.17.194:57641)
0807 05:05:05.661622 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.661637 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.661730 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.661979 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.661982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.662001 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.662151 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.662159 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.663187 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.663190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.664401 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.664411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.665455 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.665460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.665569 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.665594 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.665667 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.665742 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 05:05:05.666194 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.661432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41019 (local address 127.1.17.194:58529)
0807 05:05:05.662191 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.662206 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.662343 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.662743 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.662746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.662762 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.662882 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.662888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.663934 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.663937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.665078 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.665087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.665740 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.665744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.665855 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.665881 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.665977 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.666047 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":372,"thread_start_us":101,"threads_started":1}
I20260807 05:05:05.666185 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.661638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:58529 (local address 127.1.17.195:41019)
0807 05:05:05.661777 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.661784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.662279 (+ 495us) server_negotiation.cc:407] Connection header received
0807 05:05:05.662510 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.662514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.662564 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.662633 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.663017 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.663793 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.665191 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.665587 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.665737 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.665788 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.666008 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.666080 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:05:05.666355 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.661596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57641 (local address 127.1.17.193:39721)
0807 05:05:05.661734 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.661738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.661753 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:05:05.661815 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.661818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.661863 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.661961 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.662273 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.663068 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.664541 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.665219 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.665336 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.665391 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.666148 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.666234 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:05:05.666730 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4627a169ec274e5a815b7e832ffd5a48" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c" is_pre_election: true
I20260807 05:05:05.666764 9115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4627a169ec274e5a815b7e832ffd5a48" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c349eaf853433094b00eebbf7ed7cf" is_pre_election: true
I20260807 05:05:05.666908 9115 raft_consensus.cc:2454] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 0.
I20260807 05:05:05.666973 9406 raft_consensus.cc:2454] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 0.
I20260807 05:05:05.667320 9191 leader_election.cc:302] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5c349eaf853433094b00eebbf7ed7cf, ffd22e5d5fb443aab8dc294242ee72e5; no voters:
I20260807 05:05:05.667608 9497 raft_consensus.cc:2781] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:05.667670 9497 raft_consensus.cc:482] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:05.667713 9497 raft_consensus.cc:3037] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.669050 9497 raft_consensus.cc:504] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.669443 9497 leader_election.cc:288] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 election: Requested vote from peers d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:05.669831 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4627a169ec274e5a815b7e832ffd5a48" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c"
I20260807 05:05:05.669844 9115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4627a169ec274e5a815b7e832ffd5a48" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c349eaf853433094b00eebbf7ed7cf"
I20260807 05:05:05.669970 9406 raft_consensus.cc:3037] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.669978 9115 raft_consensus.cc:3037] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.670979 9406 raft_consensus.cc:2454] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 1.
I20260807 05:05:05.671077 9115 raft_consensus.cc:2454] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 1.
I20260807 05:05:05.671283 9193 leader_election.cc:302] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978b820670db4a679970cbb3b9a2e13c, ffd22e5d5fb443aab8dc294242ee72e5; no voters:
I20260807 05:05:05.671470 9497 raft_consensus.cc:2781] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:05.671684 9497 raft_consensus.cc:686] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 LEADER]: Becoming Leader. State: Replica: ffd22e5d5fb443aab8dc294242ee72e5, State: Running, Role: LEADER
I20260807 05:05:05.671940 9497 consensus_queue.cc:237] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.674396 8942 catalog_manager.cc:5495] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194). New cstate: current_term: 1 leader_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } health_report { overall_health: UNKNOWN } } }
W20260807 05:05:05.677152 9165 tablet.cc:2369] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 05:05:05 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079104, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.254@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:05.716256 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.706696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49382 (local address 127.1.17.254:45337)
0807 05:05:05.706840 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.706844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.707057 (+ 213us) server_negotiation.cc:407] Connection header received
0807 05:05:05.707205 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.707207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.707247 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.707317 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:05.708215 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.708727 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.709879 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.710105 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.710182 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.712883 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:05.712889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:05.712892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:05.712943 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:05.714341 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.714893 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.714896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.714897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.714987 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:05.715288 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:05.715290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:05.715291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:05.715462 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:05.715537 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.715898 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.716112 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":62}
I20260807 05:05:05.716284 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.706610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45337 (local address 127.0.0.1:49382)
0807 05:05:05.706944 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.706973 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.707108 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.707383 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.707386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.707925 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:05.708094 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.708103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.708866 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.708870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.709751 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.709763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.710230 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.710236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.710342 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.710963 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:05.710979 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:05.712694 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:05.714456 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.714467 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.714473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.714797 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.715084 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:05.715088 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:05.715090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:05.715197 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:05.715574 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:05.715583 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:05.715721 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.715857 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.716040 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":118,"threads_started":1}
I20260807 05:05:05.720528 8942 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:49382:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 05:05:05.721306 8942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:05.726282 9093 tablet_service.cc:1467] Processing CreateTablet for tablet 166ce019be2e437686d671a552c88138 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c578f2f04e6478db80cdeb5e1539dbb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:05:05.726282 9239 tablet_service.cc:1467] Processing CreateTablet for tablet 166ce019be2e437686d671a552c88138 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c578f2f04e6478db80cdeb5e1539dbb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:05:05.726461 9385 tablet_service.cc:1467] Processing CreateTablet for tablet 166ce019be2e437686d671a552c88138 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c578f2f04e6478db80cdeb5e1539dbb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 05:05:05.726709 9239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166ce019be2e437686d671a552c88138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.726709 9093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166ce019be2e437686d671a552c88138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.726862 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166ce019be2e437686d671a552c88138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:05.729841 9491 tablet_bootstrap.cc:492] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: Bootstrap starting.
I20260807 05:05:05.730983 9491 tablet_bootstrap.cc:654] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.731559 9490 tablet_bootstrap.cc:492] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: Bootstrap starting.
I20260807 05:05:05.732414 9490 tablet_bootstrap.cc:654] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.733621 9491 tablet_bootstrap.cc:492] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: No bootstrap required, opened a new log
I20260807 05:05:05.733727 9491 ts_tablet_manager.cc:1397] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 05:05:05.734108 9491 raft_consensus.cc:348] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.734226 9491 raft_consensus.cc:374] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.734272 9491 raft_consensus.cc:729] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978b820670db4a679970cbb3b9a2e13c, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.734426 9491 consensus_queue.cc:260] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.734723 9491 ts_tablet_manager.cc:1428] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:05:05.734678 9510 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.734812 9510 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:05:05.735214 9490 tablet_bootstrap.cc:492] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: No bootstrap required, opened a new log
I20260807 05:05:05.735314 9490 ts_tablet_manager.cc:1397] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 05:05:05.735687 9490 raft_consensus.cc:348] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.735795 9490 raft_consensus.cc:374] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.735834 9490 raft_consensus.cc:729] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c349eaf853433094b00eebbf7ed7cf, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.735983 9490 consensus_queue.cc:260] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.736300 9490 ts_tablet_manager.cc:1428] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:05.736243 9496 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.736388 9496 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:05:05.737233 9488 tablet_bootstrap.cc:492] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: Bootstrap starting.
I20260807 05:05:05.738289 9488 tablet_bootstrap.cc:654] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:05.740573 9488 tablet_bootstrap.cc:492] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: No bootstrap required, opened a new log
I20260807 05:05:05.740645 9488 ts_tablet_manager.cc:1397] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 05:05:05.741086 9488 raft_consensus.cc:348] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.741197 9488 raft_consensus.cc:374] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:05.741219 9488 raft_consensus.cc:729] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd22e5d5fb443aab8dc294242ee72e5, State: Initialized, Role: FOLLOWER
I20260807 05:05:05.741379 9488 consensus_queue.cc:260] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.741776 9488 ts_tablet_manager.cc:1428] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 05:05:05.741770 9497 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.741914 9497 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:05:05.843786 9496 raft_consensus.cc:482] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:05.843954 9496 raft_consensus.cc:504] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.845248 9496 leader_election.cc:288] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:05.851015 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.845670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:43429 (local address 127.1.17.194:37761)
0807 05:05:05.845816 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.845821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.846235 (+ 414us) server_negotiation.cc:407] Connection header received
0807 05:05:05.846323 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.846326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.846386 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.846485 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.846902 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.847711 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.849679 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.850295 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.850419 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.850473 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.850787 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.850880 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:05:05.851096 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.845557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:37761 (local address 127.1.17.193:43429)
0807 05:05:05.846129 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.846143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.846223 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.846517 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.846520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.846559 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.846782 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.846789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.847845 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.847849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.849547 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.849560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.850416 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.850420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.850603 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.850674 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.850753 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.850899 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":307,"spinlock_wait_cycles":20224,"thread_start_us":135,"threads_started":1}
I20260807 05:05:05.851318 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.845670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:47315 (local address 127.1.17.195:41019)
0807 05:05:05.845815 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.845821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.846171 (+ 350us) server_negotiation.cc:407] Connection header received
0807 05:05:05.846324 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.846327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.846396 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.846487 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:05.846902 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.847711 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.849679 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.850288 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.850419 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.850473 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.851090 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.851188 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:05:05.851503 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.845557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:41019 (local address 127.1.17.193:47315)
0807 05:05:05.846038 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.846056 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.846201 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.846517 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.846520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.846558 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:05.846783 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.846789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.847845 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.847849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.849555 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.849561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.850416 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.850420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.850626 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.850671 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.850753 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.850899 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":349,"spinlock_wait_cycles":24576,"thread_start_us":164,"threads_started":1}
I20260807 05:05:05.851680 9262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ce019be2e437686d671a552c88138" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" is_pre_election: true
I20260807 05:05:05.851882 9262 raft_consensus.cc:2454] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 0.
I20260807 05:05:05.852051 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ce019be2e437686d671a552c88138" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c" is_pre_election: true
I20260807 05:05:05.852195 9406 raft_consensus.cc:2454] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 0.
I20260807 05:05:05.852294 9043 leader_election.cc:302] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5c349eaf853433094b00eebbf7ed7cf, ffd22e5d5fb443aab8dc294242ee72e5; no voters:
I20260807 05:05:05.852612 9496 raft_consensus.cc:2781] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:05.852674 9496 raft_consensus.cc:482] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:05.852712 9496 raft_consensus.cc:3037] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.853785 9496 raft_consensus.cc:504] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.854133 9496 leader_election.cc:288] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 election: Requested vote from peers ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:05.854457 9262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ce019be2e437686d671a552c88138" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd22e5d5fb443aab8dc294242ee72e5"
I20260807 05:05:05.854566 9262 raft_consensus.cc:3037] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.854501 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ce019be2e437686d671a552c88138" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c"
I20260807 05:05:05.854648 9406 raft_consensus.cc:3037] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:05.855419 9262 raft_consensus.cc:2454] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 1.
I20260807 05:05:05.855654 9043 leader_election.cc:302] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5c349eaf853433094b00eebbf7ed7cf, ffd22e5d5fb443aab8dc294242ee72e5; no voters:
I20260807 05:05:05.855655 9406 raft_consensus.cc:2454] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 1.
I20260807 05:05:05.855811 9496 raft_consensus.cc:2781] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:05.856040 9496 raft_consensus.cc:686] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [term 1 LEADER]: Becoming Leader. State: Replica: d5c349eaf853433094b00eebbf7ed7cf, State: Running, Role: LEADER
I20260807 05:05:05.856295 9496 consensus_queue.cc:237] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:05.857575 9534 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: TxnStatusTablet state changed. Reason: New leader d5c349eaf853433094b00eebbf7ed7cf. Latest consensus state: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.857707 9534 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:05:05.857992 9536 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 05:05:05.858148 9536 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 05:05:05.858781 8943 catalog_manager.cc:5495] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193). New cstate: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } health_report { overall_health: UNKNOWN } } }
I20260807 05:05:05.890480 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.886368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.1.17.193:39721)
0807 05:05:05.886724 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.886727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.886738 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:05:05.886812 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.886813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.886860 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.886928 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:05.887237 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.887816 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.889032 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.889236 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.889313 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.889673 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.890091 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.890179 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.890290 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.890341 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260807 05:05:05.890506 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.886281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39721 (local address 127.0.0.1:37780)
0807 05:05:05.886438 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.886469 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.886564 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.886975 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.886978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.886989 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:05:05.887125 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.887132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.887921 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.887925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.888925 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.888937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.889336 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.889339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.889446 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.889500 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.890181 (+ 681us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.890197 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.890270 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.890356 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 05:05:05.893596 9262 raft_consensus.cc:1261] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Refusing update from remote peer d5c349eaf853433094b00eebbf7ed7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:05.893711 9406 raft_consensus.cc:1261] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Refusing update from remote peer d5c349eaf853433094b00eebbf7ed7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:05.894164 9534 consensus_queue.cc:1035] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:05.894325 9496 consensus_queue.cc:1035] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:05.895888 9497 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: TxnStatusTablet state changed. Reason: New leader d5c349eaf853433094b00eebbf7ed7cf. Latest consensus state: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.896008 9497 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:05:05.896978 9510 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: TxnStatusTablet state changed. Reason: New leader d5c349eaf853433094b00eebbf7ed7cf. Latest consensus state: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.897104 9510 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:05:05.898866 9496 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.898855 9534 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.898995 9534 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:05:05.899060 9496 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P d5c349eaf853433094b00eebbf7ed7cf: This TxnStatusTablet replica's current role is: LEADER
I20260807 05:05:05.899641 9497 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.899747 9497 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P ffd22e5d5fb443aab8dc294242ee72e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:05:05.901484 9510 tablet_replica.cc:439] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } } }
I20260807 05:05:05.901590 9510 tablet_replica.cc:442] T 166ce019be2e437686d671a552c88138 P 978b820670db4a679970cbb3b9a2e13c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 05:05:05.908314 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.904337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44284 (local address 127.1.17.194:37761)
0807 05:05:05.904460 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.904465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.904500 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:05:05.904678 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.904681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.904745 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.904874 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:05.905117 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.905925 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.906739 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.907087 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.907191 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.907399 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.907956 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.908066 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.908118 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.908172 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:05:05.919585 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.915824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39721 (local address 127.1.17.194:56385)
0807 05:05:05.916201 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.916213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.916302 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.916551 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.916553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.916559 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:05:05.916664 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.916668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.917704 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.917706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.918364 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.918373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.918925 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.918927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.919001 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.919047 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.919329 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.919334 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.919383 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.919445 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260807 05:05:05.919587 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.915923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:56385 (local address 127.1.17.193:39721)
0807 05:05:05.916049 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.916052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.916294 (+ 242us) server_negotiation.cc:407] Connection header received
0807 05:05:05.916390 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.916393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.916455 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.916536 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:05.916761 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.917599 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.918478 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.918820 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.918927 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.919170 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.919253 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.919325 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.919393 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.919453 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:05:05.929337 9115 raft_consensus.cc:1261] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Refusing update from remote peer ffd22e5d5fb443aab8dc294242ee72e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:05.929410 9406 raft_consensus.cc:1261] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Refusing update from remote peer ffd22e5d5fb443aab8dc294242ee72e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:05.929927 9521 consensus_queue.cc:1035] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:05.930164 9497 consensus_queue.cc:1035] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:05.958382 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.954085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:37761 (local address 127.1.17.193:37437)
0807 05:05:05.954814 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:05.954826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:05.954914 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:05.955137 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:05.955139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:05.955143 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 05:05:05.955234 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.955238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.956187 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.956189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.957003 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:05.957013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.957670 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.957674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:05.957768 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.957826 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.958138 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.958144 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:05:05.958194 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 05:05:05.958264 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":109,"threads_started":1}
I20260807 05:05:05.958397 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:05.954452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:37437 (local address 127.1.17.194:37761)
0807 05:05:05.954619 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:05.954625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:05.954898 (+ 273us) server_negotiation.cc:407] Connection header received
0807 05:05:05.954996 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:05.954999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:05.955053 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:05.955135 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:05.955299 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.956078 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.957125 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:05.957477 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:05.957581 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:05.957964 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:05.958052 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:05.958128 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:05.958207 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 05:05:05.958265 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 05:05:05.983695 8943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:05.983889 8943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:05.986377 8943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Sending DeleteTablet for 3 replicas of tablet 4627a169ec274e5a815b7e832ffd5a48
I20260807 05:05:05.987125 9385 tablet_service.cc:1514] Processing DeleteTablet for tablet 4627a169ec274e5a815b7e832ffd5a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:05 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:49224
I20260807 05:05:05.987473 9513 tablet_replica.cc:330] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: stopping tablet replica
I20260807 05:05:05.987609 9513 raft_consensus.cc:2229] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:05.987746 9513 raft_consensus.cc:2258] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:05.988605 9513 ts_tablet_manager.cc:1905] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:05.991659 9239 tablet_service.cc:1514] Processing DeleteTablet for tablet 4627a169ec274e5a815b7e832ffd5a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:05 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:44268
I20260807 05:05:05.991859 9515 tablet_replica.cc:330] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: stopping tablet replica
I20260807 05:05:05.991909 9093 tablet_service.cc:1514] Processing DeleteTablet for tablet 4627a169ec274e5a815b7e832ffd5a48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:05 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:37764
I20260807 05:05:05.992002 9515 raft_consensus.cc:2229] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:05:05.992090 9514 tablet_replica.cc:330] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: stopping tablet replica
I20260807 05:05:05.992193 9514 raft_consensus.cc:2229] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:05.992318 9514 raft_consensus.cc:2258] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:05.992355 9515 raft_consensus.cc:2258] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:05.992341 9513 ts_tablet_manager.cc:1918] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:05:05.992684 9513 log.cc:1196] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/4627a169ec274e5a815b7e832ffd5a48
I20260807 05:05:05.993113 9513 ts_tablet_manager.cc:1939] T 4627a169ec274e5a815b7e832ffd5a48 P 978b820670db4a679970cbb3b9a2e13c: Deleting consensus metadata
I20260807 05:05:05.993808 9514 ts_tablet_manager.cc:1905] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:05.993808 9515 ts_tablet_manager.cc:1905] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:05.993950 8931 catalog_manager.cc:4841] TS 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019): tablet 4627a169ec274e5a815b7e832ffd5a48 (table test-table [id=89ee201cf95a4952b67c373ba50e7353]) successfully deleted
I20260807 05:05:05.996536 9514 ts_tablet_manager.cc:1918] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:05:05.996536 9515 ts_tablet_manager.cc:1918] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:05:05.996639 9514 log.cc:1196] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/4627a169ec274e5a815b7e832ffd5a48
I20260807 05:05:05.996639 9515 log.cc:1196] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/4627a169ec274e5a815b7e832ffd5a48
I20260807 05:05:05.996940 9515 ts_tablet_manager.cc:1939] T 4627a169ec274e5a815b7e832ffd5a48 P ffd22e5d5fb443aab8dc294242ee72e5: Deleting consensus metadata
I20260807 05:05:05.996940 9514 ts_tablet_manager.cc:1939] T 4627a169ec274e5a815b7e832ffd5a48 P d5c349eaf853433094b00eebbf7ed7cf: Deleting consensus metadata
I20260807 05:05:05.997641 8929 catalog_manager.cc:4841] TS d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721): tablet 4627a169ec274e5a815b7e832ffd5a48 (table test-table [id=89ee201cf95a4952b67c373ba50e7353]) successfully deleted
I20260807 05:05:05.997825 8929 catalog_manager.cc:4841] TS ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761): tablet 4627a169ec274e5a815b7e832ffd5a48 (table test-table [id=89ee201cf95a4952b67c373ba50e7353]) successfully deleted
Aug 07 05:05:06 dist-test-slave-1x4g krb5kdc[8899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 05:05:06 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:06 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:05:06.029060 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:06.020896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.1.17.254:45337)
0807 05:05:06.021024 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:06.021027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:06.021393 (+ 366us) server_negotiation.cc:407] Connection header received
0807 05:05:06.021554 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:06.021557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:06.021617 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:06.021709 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:06.022585 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.023433 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.024680 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.025058 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.025159 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:06.028922 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49392: BlockingRecv error: recv got EOF from 127.0.0.1:49392 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 05:05:06 dist-test-slave-1x4g krb5kdc[8899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM
I20260807 05:05:06.041507 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:06.030751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.1.17.254:45337)
0807 05:05:06.030865 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:06.030869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:06.031301 (+ 432us) server_negotiation.cc:407] Connection header received
0807 05:05:06.031537 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:06.031541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:06.031590 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:06.031672 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:06.032502 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.033337 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.034580 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.034919 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.035022 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:06.037859 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:06.037863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:06.037866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:06.037891 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:06.039628 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:06.040102 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:06.040106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:06.040108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:06.040187 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:06.040469 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:06.040472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:06.040474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:06.040679 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:06.040840 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:06.041209 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 05:05:06.041324 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:05:06.044862 8943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:05:06.045305 8943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:06.050248 9239 tablet_service.cc:1467] Processing CreateTablet for tablet abbe5ead72d24f9fb37642f565a9f74a (DEFAULT_TABLE table=test-table [id=c6da45882c0f4818aa34fc4dec17573c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:06.050390 9093 tablet_service.cc:1467] Processing CreateTablet for tablet abbe5ead72d24f9fb37642f565a9f74a (DEFAULT_TABLE table=test-table [id=c6da45882c0f4818aa34fc4dec17573c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:06.050401 9385 tablet_service.cc:1467] Processing CreateTablet for tablet abbe5ead72d24f9fb37642f565a9f74a (DEFAULT_TABLE table=test-table [id=c6da45882c0f4818aa34fc4dec17573c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:06.050597 9239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abbe5ead72d24f9fb37642f565a9f74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:06.050633 9093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abbe5ead72d24f9fb37642f565a9f74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:06.050699 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abbe5ead72d24f9fb37642f565a9f74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:06.053067 9488 tablet_bootstrap.cc:492] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: Bootstrap starting.
I20260807 05:05:06.053339 9491 tablet_bootstrap.cc:492] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: Bootstrap starting.
I20260807 05:05:06.054082 9488 tablet_bootstrap.cc:654] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:06.054129 9491 tablet_bootstrap.cc:654] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:06.054734 9490 tablet_bootstrap.cc:492] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: Bootstrap starting.
I20260807 05:05:06.055284 9491 tablet_bootstrap.cc:492] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: No bootstrap required, opened a new log
I20260807 05:05:06.055358 9491 ts_tablet_manager.cc:1397] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:05:06.055495 9490 tablet_bootstrap.cc:654] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:06.055760 9491 raft_consensus.cc:348] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.055867 9491 raft_consensus.cc:374] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:06.055897 9491 raft_consensus.cc:729] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978b820670db4a679970cbb3b9a2e13c, State: Initialized, Role: FOLLOWER
I20260807 05:05:06.056015 9491 consensus_queue.cc:260] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.056264 9491 ts_tablet_manager.cc:1428] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:06.056493 9490 tablet_bootstrap.cc:492] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: No bootstrap required, opened a new log
I20260807 05:05:06.056577 9490 ts_tablet_manager.cc:1397] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 05:05:06.057065 9490 raft_consensus.cc:348] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.057180 9490 raft_consensus.cc:374] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:06.057225 9490 raft_consensus.cc:729] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c349eaf853433094b00eebbf7ed7cf, State: Initialized, Role: FOLLOWER
I20260807 05:05:06.057374 9490 consensus_queue.cc:260] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.057684 9490 ts_tablet_manager.cc:1428] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:06.057785 9488 tablet_bootstrap.cc:492] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: No bootstrap required, opened a new log
I20260807 05:05:06.057875 9488 ts_tablet_manager.cc:1397] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260807 05:05:06.058336 9488 raft_consensus.cc:348] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.058454 9488 raft_consensus.cc:374] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:06.058497 9488 raft_consensus.cc:729] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd22e5d5fb443aab8dc294242ee72e5, State: Initialized, Role: FOLLOWER
I20260807 05:05:06.058619 9488 consensus_queue.cc:260] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.058892 9488 ts_tablet_manager.cc:1428] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:05:06.113818 9496 raft_consensus.cc:482] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:06.113952 9496 raft_consensus.cc:504] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.114429 9496 leader_election.cc:288] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:06.114864 9404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbe5ead72d24f9fb37642f565a9f74a" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c" is_pre_election: true
I20260807 05:05:06.114877 9261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbe5ead72d24f9fb37642f565a9f74a" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" is_pre_election: true
I20260807 05:05:06.115041 9404 raft_consensus.cc:2454] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 0.
I20260807 05:05:06.115051 9261 raft_consensus.cc:2454] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 0.
I20260807 05:05:06.115339 9046 leader_election.cc:302] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978b820670db4a679970cbb3b9a2e13c, d5c349eaf853433094b00eebbf7ed7cf; no voters:
I20260807 05:05:06.115566 9496 raft_consensus.cc:2781] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:06.115628 9496 raft_consensus.cc:482] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:06.115654 9496 raft_consensus.cc:3037] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:06.116688 9496 raft_consensus.cc:504] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.117075 9496 leader_election.cc:288] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 election: Requested vote from peers ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:06.117542 9404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbe5ead72d24f9fb37642f565a9f74a" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c"
I20260807 05:05:06.117551 9261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abbe5ead72d24f9fb37642f565a9f74a" candidate_uuid: "d5c349eaf853433094b00eebbf7ed7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd22e5d5fb443aab8dc294242ee72e5"
I20260807 05:05:06.117673 9404 raft_consensus.cc:3037] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:06.117682 9261 raft_consensus.cc:3037] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:06.118891 9261 raft_consensus.cc:2454] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 1.
I20260807 05:05:06.118957 9404 raft_consensus.cc:2454] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5c349eaf853433094b00eebbf7ed7cf in term 1.
I20260807 05:05:06.119252 9046 leader_election.cc:302] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978b820670db4a679970cbb3b9a2e13c, d5c349eaf853433094b00eebbf7ed7cf; no voters:
I20260807 05:05:06.119453 9496 raft_consensus.cc:2781] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:06.119586 9496 raft_consensus.cc:686] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 1 LEADER]: Becoming Leader. State: Replica: d5c349eaf853433094b00eebbf7ed7cf, State: Running, Role: LEADER
I20260807 05:05:06.119741 9496 consensus_queue.cc:237] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.121387 8942 catalog_manager.cc:5495] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193). New cstate: current_term: 1 leader_uuid: "d5c349eaf853433094b00eebbf7ed7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } health_report { overall_health: UNKNOWN } } }
I20260807 05:05:06.141906 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:06.138465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.1.17.193:39721)
0807 05:05:06.138822 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:06.138825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:06.138836 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:05:06.138915 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:06.138918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:06.138972 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:06.139061 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:06.139310 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.139935 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.140811 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.141040 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.141175 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:06.141443 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:06.141520 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:06.141595 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:06.141713 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 05:05:06.141762 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277}
I20260807 05:05:06.144356 9404 raft_consensus.cc:1261] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Refusing update from remote peer d5c349eaf853433094b00eebbf7ed7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:06.144344 9261 raft_consensus.cc:1261] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Refusing update from remote peer d5c349eaf853433094b00eebbf7ed7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:06.144819 9496 consensus_queue.cc:1035] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:06.145071 9534 consensus_queue.cc:1035] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:06.155673 8943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:06.155828 8943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:06.158134 8943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Sending DeleteTablet for 3 replicas of tablet abbe5ead72d24f9fb37642f565a9f74a
I20260807 05:05:06.158836 9385 tablet_service.cc:1514] Processing DeleteTablet for tablet abbe5ead72d24f9fb37642f565a9f74a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:06 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:49224
I20260807 05:05:06.159025 9513 tablet_replica.cc:330] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: stopping tablet replica
I20260807 05:05:06.159142 9513 raft_consensus.cc:2229] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:06.159274 9513 raft_consensus.cc:2258] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:06.159730 8943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 05:05:06.159966 9513 ts_tablet_manager.cc:1905] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:06.160014 9093 tablet_service.cc:1514] Processing DeleteTablet for tablet abbe5ead72d24f9fb37642f565a9f74a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:06 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:37764
W20260807 05:05:06.160136 8943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:06.160178 9514 tablet_replica.cc:330] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: stopping tablet replica
I20260807 05:05:06.160319 9514 raft_consensus.cc:2229] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:05:06.160615 9514 raft_consensus.cc:2258] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:06.161310 9514 ts_tablet_manager.cc:1905] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:06.162629 9513 ts_tablet_manager.cc:1918] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:06.162703 9513 log.cc:1196] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/abbe5ead72d24f9fb37642f565a9f74a
I20260807 05:05:06.162998 9513 ts_tablet_manager.cc:1939] T abbe5ead72d24f9fb37642f565a9f74a P 978b820670db4a679970cbb3b9a2e13c: Deleting consensus metadata
I20260807 05:05:06.163628 8931 catalog_manager.cc:4841] TS 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019): tablet abbe5ead72d24f9fb37642f565a9f74a (table test-table [id=c6da45882c0f4818aa34fc4dec17573c]) successfully deleted
I20260807 05:05:06.164183 9514 ts_tablet_manager.cc:1918] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:06.164254 9514 log.cc:1196] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/abbe5ead72d24f9fb37642f565a9f74a
I20260807 05:05:06.164523 9514 ts_tablet_manager.cc:1939] T abbe5ead72d24f9fb37642f565a9f74a P d5c349eaf853433094b00eebbf7ed7cf: Deleting consensus metadata
I20260807 05:05:06.165259 8929 catalog_manager.cc:4841] TS d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721): tablet abbe5ead72d24f9fb37642f565a9f74a (table test-table [id=c6da45882c0f4818aa34fc4dec17573c]) successfully deleted
I20260807 05:05:06.165822 9238 tablet_service.cc:1467] Processing CreateTablet for tablet ab214cfa6df7442599346ad45c550eef (DEFAULT_TABLE table=test-table [id=bd992d9e94ac4ac980511d03790f49c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:06.166134 9238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab214cfa6df7442599346ad45c550eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:06.168598 9488 tablet_bootstrap.cc:492] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: Bootstrap starting.
I20260807 05:05:06.168855 9239 tablet_service.cc:1514] Processing DeleteTablet for tablet abbe5ead72d24f9fb37642f565a9f74a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:06 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:44268
I20260807 05:05:06.169045 9093 tablet_service.cc:1467] Processing CreateTablet for tablet ab214cfa6df7442599346ad45c550eef (DEFAULT_TABLE table=test-table [id=bd992d9e94ac4ac980511d03790f49c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:06.169332 9093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab214cfa6df7442599346ad45c550eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:06.169687 9488 tablet_bootstrap.cc:654] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:06.170292 9515 tablet_replica.cc:330] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: stopping tablet replica
I20260807 05:05:06.170460 9515 raft_consensus.cc:2229] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:06.170626 9515 raft_consensus.cc:2258] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:06.171036 9385 tablet_service.cc:1467] Processing CreateTablet for tablet ab214cfa6df7442599346ad45c550eef (DEFAULT_TABLE table=test-table [id=bd992d9e94ac4ac980511d03790f49c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:06.171324 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab214cfa6df7442599346ad45c550eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:06.172014 9490 tablet_bootstrap.cc:492] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf: Bootstrap starting.
I20260807 05:05:06.172847 9490 tablet_bootstrap.cc:654] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:06.173659 9488 tablet_bootstrap.cc:492] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: No bootstrap required, opened a new log
I20260807 05:05:06.173800 9488 ts_tablet_manager.cc:1397] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 05:05:06.174088 9491 tablet_bootstrap.cc:492] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: Bootstrap starting.
I20260807 05:05:06.174356 9488 raft_consensus.cc:348] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.174513 9488 raft_consensus.cc:374] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:06.174571 9488 raft_consensus.cc:729] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd22e5d5fb443aab8dc294242ee72e5, State: Initialized, Role: FOLLOWER
I20260807 05:05:06.174721 9488 consensus_queue.cc:260] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.175099 9491 tablet_bootstrap.cc:654] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:06.175108 9488 ts_tablet_manager.cc:1428] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:05:06.175772 9515 ts_tablet_manager.cc:1905] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:06.177608 9491 tablet_bootstrap.cc:492] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: No bootstrap required, opened a new log
I20260807 05:05:06.177747 9491 ts_tablet_manager.cc:1397] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 05:05:06.178059 9515 ts_tablet_manager.cc:1918] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:06.178140 9515 log.cc:1196] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/abbe5ead72d24f9fb37642f565a9f74a
I20260807 05:05:06.178224 9491 raft_consensus.cc:348] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.178354 9491 raft_consensus.cc:374] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:06.178378 9515 ts_tablet_manager.cc:1939] T abbe5ead72d24f9fb37642f565a9f74a P ffd22e5d5fb443aab8dc294242ee72e5: Deleting consensus metadata
I20260807 05:05:06.178398 9491 raft_consensus.cc:729] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978b820670db4a679970cbb3b9a2e13c, State: Initialized, Role: FOLLOWER
I20260807 05:05:06.178568 9491 consensus_queue.cc:260] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.179265 9491 ts_tablet_manager.cc:1428] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:05:06.179315 8929 catalog_manager.cc:4841] TS ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761): tablet abbe5ead72d24f9fb37642f565a9f74a (table test-table [id=c6da45882c0f4818aa34fc4dec17573c]) successfully deleted
I20260807 05:05:06.182085 9490 tablet_bootstrap.cc:492] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf: No bootstrap required, opened a new log
I20260807 05:05:06.182174 9490 ts_tablet_manager.cc:1397] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20260807 05:05:06.182591 9490 raft_consensus.cc:348] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.182708 9490 raft_consensus.cc:374] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:06.182741 9490 raft_consensus.cc:729] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c349eaf853433094b00eebbf7ed7cf, State: Initialized, Role: FOLLOWER
I20260807 05:05:06.182847 9490 consensus_queue.cc:260] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.183089 9490 ts_tablet_manager.cc:1428] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:05:06.419425 9521 raft_consensus.cc:482] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:06.419575 9521 raft_consensus.cc:504] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.420145 9521 leader_election.cc:288] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:06.420585 9115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab214cfa6df7442599346ad45c550eef" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c349eaf853433094b00eebbf7ed7cf" is_pre_election: true
I20260807 05:05:06.420614 9404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab214cfa6df7442599346ad45c550eef" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c" is_pre_election: true
I20260807 05:05:06.420724 9115 raft_consensus.cc:2454] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 0.
I20260807 05:05:06.420774 9404 raft_consensus.cc:2454] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 0.
I20260807 05:05:06.421068 9191 leader_election.cc:302] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5c349eaf853433094b00eebbf7ed7cf, ffd22e5d5fb443aab8dc294242ee72e5; no voters:
I20260807 05:05:06.421249 9521 raft_consensus.cc:2781] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:06.421309 9521 raft_consensus.cc:482] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:06.421336 9521 raft_consensus.cc:3037] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:06.422351 9521 raft_consensus.cc:504] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.422669 9521 leader_election.cc:288] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 election: Requested vote from peers d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721), 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019)
I20260807 05:05:06.423007 9404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab214cfa6df7442599346ad45c550eef" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978b820670db4a679970cbb3b9a2e13c"
I20260807 05:05:06.422982 9115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab214cfa6df7442599346ad45c550eef" candidate_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5c349eaf853433094b00eebbf7ed7cf"
I20260807 05:05:06.423132 9404 raft_consensus.cc:3037] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:06.423142 9115 raft_consensus.cc:3037] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:06.424080 9404 raft_consensus.cc:2454] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 1.
I20260807 05:05:06.424103 9115 raft_consensus.cc:2454] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd22e5d5fb443aab8dc294242ee72e5 in term 1.
I20260807 05:05:06.424367 9193 leader_election.cc:302] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978b820670db4a679970cbb3b9a2e13c, ffd22e5d5fb443aab8dc294242ee72e5; no voters:
I20260807 05:05:06.424544 9521 raft_consensus.cc:2781] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:06.424631 9521 raft_consensus.cc:686] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 LEADER]: Becoming Leader. State: Replica: ffd22e5d5fb443aab8dc294242ee72e5, State: Running, Role: LEADER
I20260807 05:05:06.424814 9521 consensus_queue.cc:237] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } }
I20260807 05:05:06.426540 8942 catalog_manager.cc:5495] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194). New cstate: current_term: 1 leader_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffd22e5d5fb443aab8dc294242ee72e5" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 37761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 } health_report { overall_health: UNKNOWN } } }
I20260807 05:05:06.476619 9577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:06.473499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.1.17.194:37761)
0807 05:05:06.473741 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:06.473745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:06.473765 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:05:06.473809 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:06.473811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:06.473848 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:06.473938 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:06.474184 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.474726 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.475531 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:06.475790 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:06.475899 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:06.476149 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:06.476230 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:06.476308 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:06.476399 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 05:05:06.476452 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":78,"threads_started":1}
I20260807 05:05:06.482393 9404 raft_consensus.cc:1261] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Refusing update from remote peer ffd22e5d5fb443aab8dc294242ee72e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:06.482642 9115 raft_consensus.cc:1261] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Refusing update from remote peer ffd22e5d5fb443aab8dc294242ee72e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:06.482846 9521 consensus_queue.cc:1035] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978b820670db4a679970cbb3b9a2e13c" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:06.483054 9547 consensus_queue.cc:1035] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5c349eaf853433094b00eebbf7ed7cf" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:06.513772 8943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:06.513937 8943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:06.515967 8943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffacf06f679f4275b7bf21d9568fb051: Sending DeleteTablet for 3 replicas of tablet ab214cfa6df7442599346ad45c550eef
I20260807 05:05:06.516561 9385 tablet_service.cc:1514] Processing DeleteTablet for tablet ab214cfa6df7442599346ad45c550eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:06 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:49224
I20260807 05:05:06.516551 9093 tablet_service.cc:1514] Processing DeleteTablet for tablet ab214cfa6df7442599346ad45c550eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:06 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:37764
I20260807 05:05:06.516793 9513 tablet_replica.cc:330] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: stopping tablet replica
I20260807 05:05:06.516896 9514 tablet_replica.cc:330] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf: stopping tablet replica
I20260807 05:05:06.516911 9513 raft_consensus.cc:2229] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:06.517028 9513 raft_consensus.cc:2258] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:06.517027 9514 raft_consensus.cc:2229] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:06.517146 9514 raft_consensus.cc:2258] T ab214cfa6df7442599346ad45c550eef P d5c349eaf853433094b00eebbf7ed7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:06.517256 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 9024
I20260807 05:05:06.517288 9239 tablet_service.cc:1514] Processing DeleteTablet for tablet ab214cfa6df7442599346ad45c550eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:06 UTC) from {username='oryx', principal='oryx/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:44268
I20260807 05:05:06.519058 9515 tablet_replica.cc:330] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: stopping tablet replica
I20260807 05:05:06.519209 9515 raft_consensus.cc:2229] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:05:06.519407 9515 raft_consensus.cc:2258] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:06.520143 9515 ts_tablet_manager.cc:1905] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:06.521868 9513 ts_tablet_manager.cc:1905] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:06.523391 9515 ts_tablet_manager.cc:1918] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:05:06.523483 9515 log.cc:1196] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/ab214cfa6df7442599346ad45c550eef
I20260807 05:05:06.523751 9515 ts_tablet_manager.cc:1939] T ab214cfa6df7442599346ad45c550eef P ffd22e5d5fb443aab8dc294242ee72e5: Deleting consensus metadata
I20260807 05:05:06.523877 9513 ts_tablet_manager.cc:1918] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 05:05:06.523968 9513 log.cc:1196] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/ab214cfa6df7442599346ad45c550eef
I20260807 05:05:06.524211 9513 ts_tablet_manager.cc:1939] T ab214cfa6df7442599346ad45c550eef P 978b820670db4a679970cbb3b9a2e13c: Deleting consensus metadata
I20260807 05:05:06.524463 8929 catalog_manager.cc:4841] TS ffd22e5d5fb443aab8dc294242ee72e5 (127.1.17.194:37761): tablet ab214cfa6df7442599346ad45c550eef (table test-table [id=bd992d9e94ac4ac980511d03790f49c5]) successfully deleted
I20260807 05:05:06.524840 8931 catalog_manager.cc:4841] TS 978b820670db4a679970cbb3b9a2e13c (127.1.17.195:41019): tablet ab214cfa6df7442599346ad45c550eef (table test-table [id=bd992d9e94ac4ac980511d03790f49c5]) successfully deleted
W20260807 05:05:06.529716 8929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:39721 (error 108)
I20260807 05:05:06.530067 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 9174
I20260807 05:05:06.530961 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:06.530576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39721 (local address 127.0.0.1:37788)
0807 05:05:06.530821 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:06.530880 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":52,"threads_started":1}
W20260807 05:05:06.531080 8929 catalog_manager.cc:4584] TS d5c349eaf853433094b00eebbf7ed7cf (127.1.17.193:39721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab214cfa6df7442599346ad45c550eef: Network error: Client connection negotiation failed: client connection to 127.1.17.193:39721: connect: Connection refused (error 111)
I20260807 05:05:06.539084 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 9318
I20260807 05:05:06.546512 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 8912
2026-08-07T05:05:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3298 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:05:06 dist-test-slave-1x4g krb5kdc[9590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:05:06 dist-test-slave-1x4g krb5kdc[9590](info): set up 2 sockets
Aug 07 05:05:06 dist-test-slave-1x4g krb5kdc[9590](info): commencing operation
krb5kdc: starting...
W20260807 05:05:07.597991 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:05:07 dist-test-slave-1x4g krb5kdc[9590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:05:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:05:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:05:07.719945 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43851
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:07.830804 9603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:07.831097 9603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:07.831197 9603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:07.835134 9603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:07.835211 9603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:07.835249 9603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:07.835299 9603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:07.835338 9603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:07.840564 9603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:07.841612 9603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:07.842622 9603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:07.851511 9609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:07.851552 9610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:07.851640 9603 server_base.cc:1034] running on GCE node
W20260807 05:05:07.851547 9612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:07.852347 9603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:07.853510 9603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:07.854699 9603 hybrid_clock.cc:648] HybridClock initialized: now 1786079107854680 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T05:05:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1319 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:05:07 dist-test-slave-1x4g krb5kdc[9617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:05:07 dist-test-slave-1x4g krb5kdc[9617](info): set up 2 sockets
Aug 07 05:05:07 dist-test-slave-1x4g krb5kdc[9617](info): commencing operation
krb5kdc: starting...
W20260807 05:05:08.913954 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:05:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:05:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:05:09.042177 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44811
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:44811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:09.151433 9631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:09.151678 9631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:09.151712 9631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:09.155367 9631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:09.155418 9631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:09.155458 9631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:09.155496 9631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:09.155531 9631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:09.160617 9631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:44811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:09.161612 9631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:09.162544 9631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:09.170709 9640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:09.170693 9631 server_base.cc:1034] running on GCE node
W20260807 05:05:09.170674 9637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:09.170674 9638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:09.171460 9631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:09.172475 9631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:09.173657 9631 hybrid_clock.cc:648] HybridClock initialized: now 1786079109173641 us; error 28 us; skew 500 ppm
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:09.176358 9631 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:05:09.177225 9631 webserver.cc:460] Webserver started at http://127.1.17.254:45461/ using document root <none> and password file <none>
I20260807 05:05:09.177738 9631 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:09.177794 9631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:09.177953 9631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:09.179956 9631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "4f00ff4b32f7cd54a22541475fbe87bf"
server_key_iv: "4f2341236c075bed4b8179b5ec43d250"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.180456 9631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "4f00ff4b32f7cd54a22541475fbe87bf"
server_key_iv: "4f2341236c075bed4b8179b5ec43d250"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.183990 9631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:05:09.186318 9647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.187460 9631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:05:09.187592 9631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "4f00ff4b32f7cd54a22541475fbe87bf"
server_key_iv: "4f2341236c075bed4b8179b5ec43d250"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.187703 9631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:09.198655 9631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:09.201272 9631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:09.201407 9631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:09.208678 9631 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:44811
I20260807 05:05:09.208707 9701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:44811 every 8 connection(s)
I20260807 05:05:09.209664 9631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:05:09.212594 9702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:09.218209 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 9631
I20260807 05:05:09.218321 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:05:09.218467 9702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Bootstrap starting.
I20260807 05:05:09.218628 1095 external_mini_cluster.cc:1428] Setting key 652ad56118dde77e880f6b6d7594ad95
I20260807 05:05:09.221478 9702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:09.222470 9702 log.cc:824] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:09.224840 9702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: No bootstrap required, opened a new log
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:05:09.229017 9702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:09.229322 9702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:09.229405 9702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4c5e44f4544a0a9c5be1f4eb54ed72, State: Initialized, Role: FOLLOWER
I20260807 05:05:09.229951 9702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:09.230149 9702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:05:09.230229 9702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:05:09.230371 9702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:09.231590 9702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:09.232013 9702 leader_election.cc:302] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e4c5e44f4544a0a9c5be1f4eb54ed72; no voters:
I20260807 05:05:09.232393 9702 leader_election.cc:288] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:05:09.232481 9707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:09.232707 9707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 LEADER]: Becoming Leader. State: Replica: 1e4c5e44f4544a0a9c5be1f4eb54ed72, State: Running, Role: LEADER
I20260807 05:05:09.233093 9707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:09.233788 9702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:09.234881 9711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e4c5e44f4544a0a9c5be1f4eb54ed72. Latest consensus state: current_term: 1 leader_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } } }
I20260807 05:05:09.235019 9711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:09.235227 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:09.219999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.1.17.254:44811)
0807 05:05:09.220443 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:09.220464 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:09.220507 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:05:09.221564 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:09.221590 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:09.222050 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:09.222454 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:09.223224 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.224870 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.225753 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.226560 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.226743 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:09.229155 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:09.229161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:09.229177 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:09.229210 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:09.231485 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.231955 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.231960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.231966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.232117 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.232557 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.232561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.232563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.233119 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:09.233327 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:09.233589 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 05:05:09.234100 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":115,"threads_started":1}
I20260807 05:05:09.235350 9709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } } }
I20260807 05:05:09.235430 9709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: This master's current role is: LEADER
W20260807 05:05:09.239285 9724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:05:09.239364 9724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:05:09.239439 9718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:05:09.240296 9718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:05:09.245857 9718 catalog_manager.cc:1347] Generated new cluster ID: 37441201a5f34a3a941fe6f6a21b3f74
I20260807 05:05:09.245922 9718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:05:09.253768 9718 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:05:09.254593 9718 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:05:09.266798 9718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Generated new TSK 0
I20260807 05:05:09.267484 9718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:05:09.305850 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44811
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:09.436290 9730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:09.436535 9730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:09.436575 9730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:09.440419 9730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:09.440469 9730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:09.440523 9730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:05:09.445632 9730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:09.446563 9730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:09.447499 9730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:09.456732 9738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:09.456707 9735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:09.456844 9730 server_base.cc:1034] running on GCE node
W20260807 05:05:09.456712 9736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:09.457410 9730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:09.458055 9730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:09.459234 9730 hybrid_clock.cc:648] HybridClock initialized: now 1786079109459206 us; error 42 us; skew 500 ppm
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:09.462015 9730 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:05:09.462819 9730 webserver.cc:460] Webserver started at http://127.1.17.193:41233/ using document root <none> and password file <none>
I20260807 05:05:09.463403 9730 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:09.463490 9730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:09.463701 9730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:09.465720 9730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "5431038af45547a6a4c86e0dbf28da71"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "6dad35a73dc235ca79c98b6436e3d664"
server_key_iv: "1e9e700a800d068d3af44071428edbe2"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.466297 9730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "5431038af45547a6a4c86e0dbf28da71"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "6dad35a73dc235ca79c98b6436e3d664"
server_key_iv: "1e9e700a800d068d3af44071428edbe2"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.469942 9730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 05:05:09.472196 9748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.473480 9730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:05:09.473618 9730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "5431038af45547a6a4c86e0dbf28da71"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "6dad35a73dc235ca79c98b6436e3d664"
server_key_iv: "1e9e700a800d068d3af44071428edbe2"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.473722 9730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:09.490272 9730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:09.493247 9730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:09.493434 9730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:09.493992 9730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:09.494772 9730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:09.494838 9730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.494911 9730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:09.494951 9730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.503772 9730 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:37143
I20260807 05:05:09.503798 9868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:37143 every 8 connection(s)
I20260807 05:05:09.504598 9730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:05:09.512740 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 9730
I20260807 05:05:09.512916 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:05:09.513216 1095 external_mini_cluster.cc:1428] Setting key 47871f8d17e81fe053e3a14e1cc9fc4e
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:05:09.518347 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:09.506312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:51875 (local address 127.1.17.254:44811)
0807 05:05:09.506472 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:09.506477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:09.507241 (+ 764us) server_negotiation.cc:407] Connection header received
0807 05:05:09.508876 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:09.508879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:09.508947 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:09.509021 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:09.510094 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.510625 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.511760 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.512015 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.512107 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:09.514656 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:09.514659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:09.514662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:09.514687 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:09.516144 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.516725 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.516729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.516731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.516868 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.517254 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.517256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.517257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.517433 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:09.517512 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:09.518124 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 05:05:09.518228 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:05:09.519538 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:09.506675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:51875)
0807 05:05:09.507104 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:09.507146 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:09.508617 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:09.509184 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:09.509190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:09.509522 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:09.509970 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:09.509982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.510752 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.510754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:09.511633 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:09.511642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.512152 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.512155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:09.512290 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:09.512904 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:09.512928 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:09.514503 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:09.516264 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:09.516270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:09.516284 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:09.516608 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:09.516962 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:09.516967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:09.516971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:09.517161 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:09.517535 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:09.517544 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:09.517871 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 05:05:09.518090 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 05:05:09.518435 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":111,"threads_started":1}
I20260807 05:05:09.520853 9869 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44811
I20260807 05:05:09.521106 9869 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:09.521608 9869 heartbeater.cc:511] Master 127.1.17.254:44811 requested a full tablet report, sending...
I20260807 05:05:09.523135 9664 ts_manager.cc:194] Registered new tserver with Master: 5431038af45547a6a4c86e0dbf28da71 (127.1.17.193:37143)
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:05:09.524290 9664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:51875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:05:09.541527 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44811
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:09.650605 9874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:09.650825 9874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:09.650863 9874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:09.654762 9874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:09.654809 9874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:09.654883 9874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:05:09.659987 9874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:09.661000 9874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:09.661962 9874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:09.671355 9879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:09.671330 9882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:09.671317 9880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:09.671633 9874 server_base.cc:1034] running on GCE node
I20260807 05:05:09.672135 9874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:09.672714 9874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:09.673930 9874 hybrid_clock.cc:648] HybridClock initialized: now 1786079109673915 us; error 25 us; skew 500 ppm
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:09.676766 9874 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:05:09.677711 9874 webserver.cc:460] Webserver started at http://127.1.17.194:38845/ using document root <none> and password file <none>
I20260807 05:05:09.678274 9874 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:09.678362 9874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:09.678608 9874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:09.680626 9874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "d73e39e9c1e24e07a47b9e04f0d80b21"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "2cc8b03124aa25a8f66f970b4a505291"
server_key_iv: "0a0f88abe035e25e4372bdd6b43cda16"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.681236 9874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "d73e39e9c1e24e07a47b9e04f0d80b21"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "2cc8b03124aa25a8f66f970b4a505291"
server_key_iv: "0a0f88abe035e25e4372bdd6b43cda16"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.684861 9874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:05:09.687345 9889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.688529 9874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:09.688650 9874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "d73e39e9c1e24e07a47b9e04f0d80b21"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "2cc8b03124aa25a8f66f970b4a505291"
server_key_iv: "0a0f88abe035e25e4372bdd6b43cda16"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.688747 9874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:09.721685 9874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:09.724881 9874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:09.725087 9874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:09.725698 9874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:09.726529 9874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:09.726576 9874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.726643 9874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:09.726675 9874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.735247 9874 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:42953
I20260807 05:05:09.735287 10013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:42953 every 8 connection(s)
I20260807 05:05:09.736189 9874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:05:09.737528 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 9874
I20260807 05:05:09.737654 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:05:09.737968 1095 external_mini_cluster.cc:1428] Setting key 06e29a1b0e800f82dc45bd21607a78bb
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:05:09.750783 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:09.738124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:53611 (local address 127.1.17.254:44811)
0807 05:05:09.738262 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:09.738266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:09.739505 (+ 1239us) server_negotiation.cc:407] Connection header received
0807 05:05:09.740310 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:09.740316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:09.740391 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:09.740478 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:09.741843 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.742363 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.744100 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.744341 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.744428 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:09.747279 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:09.747282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:09.747285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:09.747314 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:09.748657 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.749182 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.749187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.749188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.749268 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.749632 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.749635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.749637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.749878 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:09.750006 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:09.750480 (+ 474us) server_negotiation.cc:299] Negotiation successful
0807 05:05:09.750615 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
I20260807 05:05:09.751564 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:09.738532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:53611)
0807 05:05:09.739333 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:09.739386 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:09.740111 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:09.740621 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:09.740627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:09.740995 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:09.741673 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:09.741697 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.742494 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.742501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:09.743942 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:09.743957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.744528 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.744535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:09.744759 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:09.745371 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:09.745396 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:09.747126 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:09.748798 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:09.748806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:09.748820 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:09.749072 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:09.749361 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:09.749364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:09.749366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:09.749530 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:09.750006 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:09.750013 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:09.750244 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 05:05:09.750421 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 05:05:09.750744 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":125,"threads_started":1}
I20260807 05:05:09.752677 10014 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44811
I20260807 05:05:09.752969 10014 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:09.753484 10014 heartbeater.cc:511] Master 127.1.17.254:44811 requested a full tablet report, sending...
I20260807 05:05:09.754539 9664 ts_manager.cc:194] Registered new tserver with Master: d73e39e9c1e24e07a47b9e04f0d80b21 (127.1.17.194:42953)
I20260807 05:05:09.755106 9664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:53611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:05:09.767416 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44811
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:09.878840 10019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:09.879073 10019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:09.879141 10019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:09.883028 10019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:09.883078 10019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:09.883178 10019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:05:09.888171 10019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:09.889168 10019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:09.890187 10019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:09.899710 10029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:09.899724 10027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:09.899719 10026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:09.900019 10019 server_base.cc:1034] running on GCE node
I20260807 05:05:09.900413 10019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:09.901026 10019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:09.902195 10019 hybrid_clock.cc:648] HybridClock initialized: now 1786079109902184 us; error 30 us; skew 500 ppm
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:09.904904 10019 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:05:09.905736 10019 webserver.cc:460] Webserver started at http://127.1.17.195:43201/ using document root <none> and password file <none>
I20260807 05:05:09.906288 10019 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:09.906373 10019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:09.906577 10019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:09.908494 10019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "4cfd08e122174a5098dbe532cd4c2e8b"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "bc924ae900399c44461423f35123811d"
server_key_iv: "93a0a6b08c6a69c5876eb67c46ed94b2"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.909054 10019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "4cfd08e122174a5098dbe532cd4c2e8b"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "bc924ae900399c44461423f35123811d"
server_key_iv: "93a0a6b08c6a69c5876eb67c46ed94b2"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.912410 10019 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:05:09.914769 10036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.915901 10019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:05:09.916033 10019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "4cfd08e122174a5098dbe532cd4c2e8b"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "bc924ae900399c44461423f35123811d"
server_key_iv: "93a0a6b08c6a69c5876eb67c46ed94b2"
server_key_version: "encryptionkey@0"
I20260807 05:05:09.916144 10019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:09.928841 10019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:09.931316 10019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:09.931491 10019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:09.932010 10019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:09.932761 10019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:09.932852 10019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.932915 10019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:09.932962 10019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:09.942014 10019 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:38851
I20260807 05:05:09.942045 10157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:38851 every 8 connection(s)
I20260807 05:05:09.942811 10019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:05:09.943490 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10019
I20260807 05:05:09.943625 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:05:09.943935 1095 external_mini_cluster.cc:1428] Setting key 96b860c32a13b66e6c3e09d97b09ab37
Aug 07 05:05:09 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:05:09.956095 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:09.944701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:34391 (local address 127.1.17.254:44811)
0807 05:05:09.944889 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:09.944895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:09.945849 (+ 954us) server_negotiation.cc:407] Connection header received
0807 05:05:09.946725 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:09.946730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:09.946786 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:09.946872 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:09.948443 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.949337 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.950636 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.950838 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.950916 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:09.953067 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:09.953070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:09.953072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:09.953095 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:09.954480 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.954894 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.954896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.954897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.954971 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:09.955243 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:09.955245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:09.955246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:09.955399 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:09.955481 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:09.955863 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 05:05:09.955979 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:05:09.956912 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:09.945038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:34391)
0807 05:05:09.945711 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:09.945761 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:09.946531 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:09.947129 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:09.947138 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:09.947637 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:09.948265 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:09.948285 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.949496 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.949502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:09.950531 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:09.950541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:09.950954 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:09.950957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:09.951089 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:09.951529 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:09.951543 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:09.952919 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:09.954578 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:09.954583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:09.954593 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:09.954816 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:09.955043 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:09.955045 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:09.955047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:09.955171 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:09.955480 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:09.955485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:09.955680 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 05:05:09.955817 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 05:05:09.956052 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":125,"threads_started":1}
I20260807 05:05:09.957892 10158 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44811
I20260807 05:05:09.958082 10158 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:09.958483 10158 heartbeater.cc:511] Master 127.1.17.254:44811 requested a full tablet report, sending...
I20260807 05:05:09.959383 9663 ts_manager.cc:194] Registered new tserver with Master: 4cfd08e122174a5098dbe532cd4c2e8b (127.1.17.195:38851)
I20260807 05:05:09.959924 9663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:34391
I20260807 05:05:09.969017 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab.
Entry for principal oryx/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab.
WARNING: no policy specified for oryx/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab.
Entry for principal oryx/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab.
WARNING: no policy specified for oryx/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab.
Entry for principal oryx/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab.
I20260807 05:05:10.040287 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 9730
I20260807 05:05:10.048334 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 9874
I20260807 05:05:10.054190 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10019
I20260807 05:05:10.060226 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 9631
I20260807 05:05:10.067580 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44811
--webserver_interface=127.1.17.254
--webserver_port=45461
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:44811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:10.181756 10168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:10.182010 10168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:10.182051 10168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:10.185842 10168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:10.185925 10168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:10.185947 10168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:10.185966 10168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:10.185985 10168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:10.190834 10168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:44811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=45461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:10.191794 10168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:10.192752 10168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:10.201164 10176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:10.201220 10168 server_base.cc:1034] running on GCE node
W20260807 05:05:10.201181 10173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:10.201172 10174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:10.201992 10168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:10.203083 10168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:10.204242 10168 hybrid_clock.cc:648] HybridClock initialized: now 1786079110204212 us; error 45 us; skew 500 ppm
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:10.207011 10168 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:05:10.207777 10168 webserver.cc:460] Webserver started at http://127.1.17.254:45461/ using document root <none> and password file <none>
I20260807 05:05:10.208267 10168 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:10.208333 10168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:10.212023 10168 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 05:05:10.214148 10183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.215363 10168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:05:10.215493 10168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "4f00ff4b32f7cd54a22541475fbe87bf"
server_key_iv: "4f2341236c075bed4b8179b5ec43d250"
server_key_version: "encryptionkey@0"
I20260807 05:05:10.216042 10168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:10.230726 10168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:10.233386 10168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:10.233573 10168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:10.240751 10168 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:44811
I20260807 05:05:10.240765 10237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:44811 every 8 connection(s)
I20260807 05:05:10.241817 10168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:05:10.243963 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10168
I20260807 05:05:10.244437 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:37143
--local_ip_for_outbound_sockets=127.1.17.193
--tserver_master_addrs=127.1.17.254:44811
--webserver_port=41233
--webserver_interface=127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:05:10.246721 10238 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:05:10.248325 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Bootstrap starting.
I20260807 05:05:10.265827 10238 log.cc:824] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:10.272193 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:05:10.272583 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Bootstrap complete.
I20260807 05:05:10.276996 10238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:10.277398 10238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4c5e44f4544a0a9c5be1f4eb54ed72, State: Initialized, Role: FOLLOWER
I20260807 05:05:10.278013 10238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:10.278156 10238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:05:10.278226 10238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:05:10.278339 10238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:05:10.280325 10238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:10.280658 10238 leader_election.cc:302] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e4c5e44f4544a0a9c5be1f4eb54ed72; no voters:
I20260807 05:05:10.281008 10238 leader_election.cc:288] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:05:10.281095 10242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:05:10.281309 10242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [term 2 LEADER]: Becoming Leader. State: Replica: 1e4c5e44f4544a0a9c5be1f4eb54ed72, State: Running, Role: LEADER
I20260807 05:05:10.281597 10242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } }
I20260807 05:05:10.282263 10238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:10.282522 10244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } } }
I20260807 05:05:10.282644 10244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:10.282990 10243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e4c5e44f4544a0a9c5be1f4eb54ed72. Latest consensus state: current_term: 2 leader_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4c5e44f4544a0a9c5be1f4eb54ed72" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44811 } } }
I20260807 05:05:10.283108 10243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:10.283471 10247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:05:10.286792 10247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:05:10.287495 10247 catalog_manager.cc:1259] Loaded cluster ID: 37441201a5f34a3a941fe6f6a21b3f74
I20260807 05:05:10.287568 10247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:05:10.289350 10247 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:05:10.290061 10247 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1e4c5e44f4544a0a9c5be1f4eb54ed72: Loaded TSK: 0
I20260807 05:05:10.290980 10247 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:05:10.367467 10240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:10.367698 10240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:10.367738 10240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:10.371556 10240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:10.371610 10240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:10.371686 10240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:05:10.376852 10240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.193.keytab
--principal=oryx/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:37143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=41233
--tserver_master_addrs=127.1.17.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:10.377838 10240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:10.378851 10240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:10.388573 10265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:10.388568 10266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:10.388955 10240 server_base.cc:1034] running on GCE node
W20260807 05:05:10.389204 10268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:10.389617 10240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:10.390323 10240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:10.391551 10240 hybrid_clock.cc:648] HybridClock initialized: now 1786079110391522 us; error 58 us; skew 500 ppm
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079110, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:10.394729 10240 init.cc:377] Logged in from keytab as oryx/127.1.17.193@KRBTEST.COM (short username oryx)
I20260807 05:05:10.395785 10240 webserver.cc:460] Webserver started at http://127.1.17.193:41233/ using document root <none> and password file <none>
I20260807 05:05:10.396376 10240 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:10.396479 10240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:10.400677 10240 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:05:10.402920 10275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.404193 10240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:10.404285 10240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "5431038af45547a6a4c86e0dbf28da71"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "6dad35a73dc235ca79c98b6436e3d664"
server_key_iv: "1e9e700a800d068d3af44071428edbe2"
server_key_version: "encryptionkey@0"
I20260807 05:05:10.404903 10240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:10.422276 10240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:10.425549 10240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:10.425746 10240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:10.426309 10240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:10.427104 10240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:10.427172 10240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.427240 10240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:10.427280 10240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.435585 10240 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:37143
I20260807 05:05:10.435842 10394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:37143 every 8 connection(s)
I20260807 05:05:10.436666 10240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:05:10.442132 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10240
I20260807 05:05:10.442564 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:42953
--local_ip_for_outbound_sockets=127.1.17.194
--tserver_master_addrs=127.1.17.254:44811
--webserver_port=38845
--webserver_interface=127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:05:10.451460 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.438970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:38711 (local address 127.1.17.254:44811)
0807 05:05:10.439543 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.439566 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.439954 (+ 388us) server_negotiation.cc:407] Connection header received
0807 05:05:10.441129 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.441142 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.441439 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.441792 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.443163 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.444203 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.445605 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.446257 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.446400 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.450726 (+ 4326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:38711: BlockingRecv error: recv got EOF from 127.1.17.193:38711 (error 108)
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":145,"threads_started":1}
I20260807 05:05:10.451660 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.439061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:38711)
0807 05:05:10.439772 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.439812 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.440626 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.441883 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.441892 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.442405 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.442973 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.442991 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.444416 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.444423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.445462 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.445474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.446456 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.446463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.446697 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.447358 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.447389 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.450541 (+ 3152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":133,"threads_started":1}
W20260807 05:05:10.452297 10395 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:44811 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.17.254:44811: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:10.459918 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.452727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:50981)
0807 05:05:10.452945 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.452960 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.453046 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.453622 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.453627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.453801 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.453929 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.453937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.455031 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.455036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.456031 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.456047 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.456756 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.456762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.456910 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.457484 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.457505 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.459791 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133}
W20260807 05:05:10.460429 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.453172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:50981 (local address 127.1.17.254:44811)
0807 05:05:10.453347 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.453352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.453369 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:05:10.453414 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.453428 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.453482 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.453571 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.454074 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.454892 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.456210 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.456612 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.456753 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.459980 (+ 3227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:50981: BlockingRecv error: recv got EOF from 127.1.17.193:50981 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:10.466457 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.460626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:40335)
0807 05:05:10.460918 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.460931 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.461013 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.461296 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.461299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.461479 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.461653 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.461660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.462585 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.462589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.463434 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.463444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.463828 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.463831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.463911 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.464365 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.464379 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.466299 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227}
W20260807 05:05:10.466485 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.460690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:40335 (local address 127.1.17.254:44811)
0807 05:05:10.460827 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.460833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.461069 (+ 236us) server_negotiation.cc:407] Connection header received
0807 05:05:10.461117 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.461121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.461176 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.461285 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.461788 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.462465 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.463540 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.463731 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.463803 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.466408 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:40335: BlockingRecv error: recv got EOF from 127.1.17.193:40335 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 05:05:10.466655 10395 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:05:10.563787 10399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:10.564105 10399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:10.564186 10399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:10.568150 10399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:10.568233 10399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:10.568320 10399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:05:10.573485 10399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.194.keytab
--principal=oryx/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:42953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=38845
--tserver_master_addrs=127.1.17.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:10.574726 10399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:10.575922 10399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:10.585356 10405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:10.585350 10407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:10.585520 10399 server_base.cc:1034] running on GCE node
W20260807 05:05:10.585776 10404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:10.586225 10399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:10.586822 10399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:10.588006 10399 hybrid_clock.cc:648] HybridClock initialized: now 1786079110587985 us; error 30 us; skew 500 ppm
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079110, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:10.591125 10399 init.cc:377] Logged in from keytab as oryx/127.1.17.194@KRBTEST.COM (short username oryx)
I20260807 05:05:10.591902 10399 webserver.cc:460] Webserver started at http://127.1.17.194:38845/ using document root <none> and password file <none>
I20260807 05:05:10.592396 10399 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:10.592474 10399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:10.596216 10399 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:10.598356 10414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.599558 10399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:05:10.599654 10399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "d73e39e9c1e24e07a47b9e04f0d80b21"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "2cc8b03124aa25a8f66f970b4a505291"
server_key_iv: "0a0f88abe035e25e4372bdd6b43cda16"
server_key_version: "encryptionkey@0"
I20260807 05:05:10.600165 10399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:10.609727 10399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:10.612888 10399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:10.613037 10399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:10.613546 10399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:10.614288 10399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:10.614333 10399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.614372 10399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:10.614387 10399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.623358 10399 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:42953
I20260807 05:05:10.623397 10532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:42953 every 8 connection(s)
I20260807 05:05:10.624348 10399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:05:10.630613 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10399
I20260807 05:05:10.631044 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:38851
--local_ip_for_outbound_sockets=127.1.17.195
--tserver_master_addrs=127.1.17.254:44811
--webserver_port=43201
--webserver_interface=127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:05:10.635501 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.626099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:60855 (local address 127.1.17.254:44811)
0807 05:05:10.626241 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.626245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.627306 (+ 1061us) server_negotiation.cc:407] Connection header received
0807 05:05:10.628160 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.628162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.628210 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.628274 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.629535 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.630047 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.631425 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.631636 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.631737 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.635367 (+ 3630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:60855: BlockingRecv error: recv got EOF from 127.1.17.194:60855 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:05:10.636092 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.626502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:60855)
0807 05:05:10.627171 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.627211 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.627970 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.628441 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.628447 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.628931 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.629404 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.629420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.630201 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.630204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.631279 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.631290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.631790 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.631794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.631970 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.632574 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.632598 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.635249 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":111,"threads_started":1}
W20260807 05:05:10.636677 10533 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:44811 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.17.254:44811: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:10.642163 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.637151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:52771)
0807 05:05:10.637252 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.637268 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.637340 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.637569 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.637572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.637724 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.637818 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.637822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.638528 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.638533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.639112 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.639120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.639553 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.639557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.639644 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.640092 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.640103 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.642026 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 05:05:10.642254 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.637242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:52771 (local address 127.1.17.254:44811)
0807 05:05:10.637353 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.637358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.637383 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:05:10.637438 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.637441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.637476 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.637554 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.637951 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.638427 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.639231 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.639448 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.639550 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.642135 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:52771: BlockingRecv error: recv got EOF from 127.1.17.194:52771 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:10.648094 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.642937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:34763)
0807 05:05:10.643079 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.643097 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.643181 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.643413 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.643417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.643607 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.643696 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.643702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.644369 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.644372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.645250 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.645260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.645709 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.645713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.645814 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.646291 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.646303 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.647964 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 05:05:10.648188 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.643043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:34763 (local address 127.1.17.254:44811)
0807 05:05:10.643146 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.643150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.643163 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:10.643269 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.643271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.643303 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.643366 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.643795 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.644272 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.645385 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.645598 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.645687 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.648073 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:34763: BlockingRecv error: recv got EOF from 127.1.17.194:34763 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 05:05:10.648357 10533 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:05:10.746368 10536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:10.746635 10536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:10.746701 10536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:10.750593 10536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:10.750669 10536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:10.750747 10536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:05:10.755761 10536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/krb5kdc/oryx_127.1.17.195.keytab
--principal=oryx/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:38851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=43201
--tserver_master_addrs=127.1.17.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:10.756767 10536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:10.757865 10536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:10.767051 10544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:10.767055 10542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:10.767095 10541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:10.767230 10536 server_base.cc:1034] running on GCE node
I20260807 05:05:10.767774 10536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:10.768380 10536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:10.769534 10536 hybrid_clock.cc:648] HybridClock initialized: now 1786079110769519 us; error 26 us; skew 500 ppm
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079110, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:10.772326 10536 init.cc:377] Logged in from keytab as oryx/127.1.17.195@KRBTEST.COM (short username oryx)
I20260807 05:05:10.773218 10536 webserver.cc:460] Webserver started at http://127.1.17.195:43201/ using document root <none> and password file <none>
I20260807 05:05:10.773761 10536 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:10.773854 10536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:10.777649 10536 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:05:10.779685 10551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.780925 10536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:05:10.781050 10536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "4cfd08e122174a5098dbe532cd4c2e8b"
format_stamp: "Formatted at 2026-08-07 05:05:09 on dist-test-slave-1x4g"
server_key: "bc924ae900399c44461423f35123811d"
server_key_iv: "93a0a6b08c6a69c5876eb67c46ed94b2"
server_key_version: "encryptionkey@0"
I20260807 05:05:10.781587 10536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:10.816184 10536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:10.819706 10536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:10.819926 10536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:10.820533 10536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:10.821412 10536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:10.821478 10536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.821544 10536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:10.821588 10536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:10.831660 10536 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:38851
I20260807 05:05:10.831717 10673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:38851 every 8 connection(s)
I20260807 05:05:10.833679 10536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:05:10.839123 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10536
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 05:05:10.845889 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.835794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:47837 (local address 127.1.17.254:44811)
0807 05:05:10.835934 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.835938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.837075 (+ 1137us) server_negotiation.cc:407] Connection header received
0807 05:05:10.837993 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.837997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.838054 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.838167 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.839347 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.840055 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.841466 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.841679 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.841753 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.845741 (+ 3988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:47837: BlockingRecv error: recv got EOF from 127.1.17.195:47837 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:05:10.846676 10676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.836194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:47837)
0807 05:05:10.836914 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.836979 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.837772 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.838343 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.838349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.838694 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.839187 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.839209 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.840186 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.840190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.841352 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.841362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.841816 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.841819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.842040 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.842677 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.842694 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.845617 (+ 2923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":124,"threads_started":1}
W20260807 05:05:10.847170 10674 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:44811 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.17.254:44811: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
I20260807 05:05:10.849006 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.840118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.1.17.254:44811)
0807 05:05:10.840343 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.840347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.840360 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:10.840395 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.840397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.840437 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.840535 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.841309 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.841955 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.842803 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.843142 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.843263 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.844740 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:10.844751 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:10.844770 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:10.844844 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:10.846857 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:10.847353 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:10.847361 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:10.847368 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:10.847526 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:10.847803 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:10.847806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:10.847807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:10.848126 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:10.848321 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:10.848623 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 05:05:10.848891 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":69,"threads_started":1}
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:10.852811 10676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.847553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:34471)
0807 05:05:10.847867 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.847877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.847965 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.848189 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.848192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.848340 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.848423 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.848428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.849287 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.849291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.850066 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.850076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.850559 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.850561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.850625 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.851043 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.851055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.852656 (+ 1601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256}
W20260807 05:05:10.852891 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.847639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:34471 (local address 127.1.17.254:44811)
0807 05:05:10.847749 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.847753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.847933 (+ 180us) server_negotiation.cc:407] Connection header received
0807 05:05:10.848055 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.848057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.848099 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.848173 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.848534 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.849186 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.850168 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.850460 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.850560 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.852757 (+ 2197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:34471: BlockingRecv error: recv got EOF from 127.1.17.195:34471 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:10 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:10.857462 10676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:10.853340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:42061)
0807 05:05:10.853480 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:10.853496 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:10.853564 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:10.853737 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:10.853739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:10.853853 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:10.853917 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.853922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.854538 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.854540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.855026 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:10.855032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.855401 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.855403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:10.855461 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.855789 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:10.855799 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:10.857375 (+ 1576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 05:05:10.857628 10397 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:10.853412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:42061 (local address 127.1.17.254:44811)
0807 05:05:10.853549 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:10.853552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:10.853564 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:05:10.853621 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:10.853623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:10.853652 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:10.853712 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:10.854005 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.854443 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.855114 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:10.855303 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:10.855372 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:10.857508 (+ 2136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:42061: BlockingRecv error: recv got EOF from 127.1.17.195:42061 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 05:05:10.857729 10674 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 05:05:11 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:11 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:11.472724 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:11.467313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:50471)
0807 05:05:11.467646 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:11.467662 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:11.467785 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:11.468088 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:11.468092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:11.468283 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:11.468384 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:11.468389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.469246 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.469249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:11.469828 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:11.469835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.470238 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.470241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:11.470302 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:11.470843 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:11.470854 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:11.472606 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":101,"threads_started":1}
W20260807 05:05:11.472975 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:11.467388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:50471 (local address 127.1.17.254:44811)
0807 05:05:11.467723 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:11.467729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:11.467744 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:05:11.467902 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:11.467905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:11.467961 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:11.468061 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:11.468496 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.469153 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.469930 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.470144 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.470218 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:11.472745 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:50471: BlockingRecv error: recv got EOF from 127.1.17.193:50471 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":83,"threads_started":1}
Aug 07 05:05:11 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:11 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:11.654587 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:11.649103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:54261)
0807 05:05:11.649382 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:11.649396 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:11.649490 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:11.649715 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:11.649717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:11.649925 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:11.650018 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:11.650024 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.650932 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.650936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:11.651627 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:11.651637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.652038 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.652041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:11.652100 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:11.652556 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:11.652567 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:11.654456 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":68,"threads_started":1}
W20260807 05:05:11.654748 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:11.649182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:54261 (local address 127.1.17.254:44811)
0807 05:05:11.649372 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:11.649376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:11.649472 (+ 96us) server_negotiation.cc:407] Connection header received
0807 05:05:11.649568 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:11.649570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:11.649611 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:11.649678 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:11.650130 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.650838 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.651728 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.651935 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.652018 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:11.654593 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:54261: BlockingRecv error: recv got EOF from 127.1.17.194:54261 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 05:05:11 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:11 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:11.864073 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:11.858387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:39739)
0807 05:05:11.858683 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:11.858698 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:11.858836 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:11.859121 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:11.859124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:11.859374 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:11.859470 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:11.859476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.860221 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.860224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:11.860862 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:11.860875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.861365 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.861368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:11.861452 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:11.862010 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:11.862022 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:11.863912 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":87,"threads_started":1}
W20260807 05:05:11.864224 10681 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:11.858480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:39739 (local address 127.1.17.254:44811)
0807 05:05:11.858637 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:11.858642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:11.858773 (+ 131us) server_negotiation.cc:407] Connection header received
0807 05:05:11.858947 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:11.858950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:11.858996 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:11.859073 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:11.859585 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.860111 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.860988 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:11.861240 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:11.861361 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:11.864101 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:39739: BlockingRecv error: recv got EOF from 127.1.17.195:39739 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 05:05:12 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:12 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:12.479269 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:12.473650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:36273)
0807 05:05:12.473925 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:12.473941 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:12.474062 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:12.474350 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:12.474353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:12.474680 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:12.474818 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:12.474824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.475618 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.475621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:12.476232 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:12.476240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.476666 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.476669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:12.476733 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:12.477218 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:12.477230 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:12.479131 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
W20260807 05:05:12.479516 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:12.473726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:36273 (local address 127.1.17.254:44811)
0807 05:05:12.474023 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:12.474027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:12.474041 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:12.474147 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:12.474151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:12.474205 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:12.474297 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:12.474964 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.475520 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.476336 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.476560 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.476648 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:12.479263 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:36273: BlockingRecv error: recv got EOF from 127.1.17.193:36273 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":106,"threads_started":1}
Aug 07 05:05:12 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:12 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:12.661252 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:12.655372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:48207)
0807 05:05:12.655714 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:12.655728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:12.655866 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:12.656187 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:12.656189 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:12.656466 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:12.656619 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:12.656629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.657469 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.657472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:12.658069 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:12.658076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.658546 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.658550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:12.658623 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:12.659098 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:12.659110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:12.661106 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":108,"threads_started":1}
W20260807 05:05:12.661337 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:12.655499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:48207 (local address 127.1.17.254:44811)
0807 05:05:12.655666 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:12.655671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:12.655821 (+ 150us) server_negotiation.cc:407] Connection header received
0807 05:05:12.655989 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:12.655995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:12.656054 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:12.656168 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:12.656752 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.657353 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.658192 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.658402 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.658492 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:12.661242 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:48207: BlockingRecv error: recv got EOF from 127.1.17.194:48207 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 05:05:12 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:12 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:12.874665 10687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:12.864863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:43497)
0807 05:05:12.865115 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:12.865128 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:12.865211 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:12.865454 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:12.865457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:12.865640 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:12.865755 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:12.865760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.866552 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.866554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:12.867108 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:12.867116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.867541 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.867543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:12.867602 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:12.872505 (+ 4903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:12.872518 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:12.874474 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
W20260807 05:05:12.874693 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:12.865031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:43497 (local address 127.1.17.254:44811)
0807 05:05:12.865179 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:12.865182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:12.865197 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:05:12.865295 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:12.865297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:12.865367 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:12.865445 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:12.865874 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.866447 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.867224 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:12.867446 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:12.867526 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:12.874594 (+ 7068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:43497: BlockingRecv error: recv got EOF from 127.1.17.195:43497 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 05:05:13 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:13 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:13.486341 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:13.480167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:42461)
0807 05:05:13.480511 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:13.480527 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:13.480624 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:13.480933 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:13.480935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:13.481148 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:13.481258 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:13.481264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.482006 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.482010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:13.482619 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:13.482629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.483153 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.483156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:13.483281 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:13.483912 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:13.483926 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:13.486149 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":145,"threads_started":1}
W20260807 05:05:13.486393 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:13.480317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:42461 (local address 127.1.17.254:44811)
0807 05:05:13.480560 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:13.480564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:13.480578 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:13.480715 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:13.480718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:13.480762 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:13.480875 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:13.481366 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.481895 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.482765 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.483040 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.483136 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:13.486272 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:42461: BlockingRecv error: recv got EOF from 127.1.17.193:42461 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
Aug 07 05:05:13 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:13 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:13.667444 10690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:13.662136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:36733)
0807 05:05:13.662473 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:13.662486 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:13.662581 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:13.662822 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:13.662824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:13.663029 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:13.663149 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:13.663154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.663914 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.663917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:13.664480 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:13.664488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.664939 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.664941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:13.665003 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:13.665457 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:13.665469 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:13.667312 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":93,"threads_started":1}
W20260807 05:05:13.667639 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:13.662250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:36733 (local address 127.1.17.254:44811)
0807 05:05:13.662388 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:13.662392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:13.662553 (+ 161us) server_negotiation.cc:407] Connection header received
0807 05:05:13.662681 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:13.662684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:13.662727 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:13.662794 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:13.663248 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.663784 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.664585 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.664835 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.664908 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:13.667469 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:36733: BlockingRecv error: recv got EOF from 127.1.17.194:36733 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 05:05:13 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:13 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:13.881390 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:13.875458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:54541)
0807 05:05:13.875846 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:13.875859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:13.875976 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:13.876260 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:13.876263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:13.876453 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:13.876575 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:13.876581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.877576 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.877580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:13.878300 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:13.878311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.878739 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.878742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:13.878803 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:13.879337 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:13.879352 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:13.881260 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":212,"threads_started":1}
W20260807 05:05:13.881542 10689 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:13.875543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:54541 (local address 127.1.17.254:44811)
0807 05:05:13.875690 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:13.875694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:13.875923 (+ 229us) server_negotiation.cc:407] Connection header received
0807 05:05:13.876078 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:13.876080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:13.876126 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:13.876216 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:13.876715 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.877451 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.878419 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:13.878635 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:13.878715 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:13.881422 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:54541: BlockingRecv error: recv got EOF from 127.1.17.195:54541 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 05:05:14 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:14 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:14.492507 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:14.487121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:52735)
0807 05:05:14.487443 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:14.487457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:14.487555 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:14.487799 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:14.487801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:14.487986 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:14.488090 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:14.488095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.488881 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.488886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:14.489512 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:14.489522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.489932 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.489934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:14.489996 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:14.490566 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:14.490577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:14.492383 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":74,"threads_started":1}
W20260807 05:05:14.492651 10693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:14.487242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:52735 (local address 127.1.17.254:44811)
0807 05:05:14.487504 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:14.487507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:14.487521 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:14.487638 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:14.487641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:14.487683 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:14.487758 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:14.488192 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.488731 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.489618 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.489834 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.489910 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:14.492519 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:52735: BlockingRecv error: recv got EOF from 127.1.17.193:52735 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
Aug 07 05:05:14 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:14 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:14.673987 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:14.668223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:48923)
0807 05:05:14.668553 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:14.668566 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:14.668708 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:14.669043 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:14.669046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:14.669257 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:14.669354 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:14.669360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.670158 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.670163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:14.670718 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:14.670726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.671189 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.671193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:14.671281 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:14.671896 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:14.671907 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:14.673817 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":113,"threads_started":1}
W20260807 05:05:14.674058 10693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:14.668288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:48923 (local address 127.1.17.254:44811)
0807 05:05:14.668452 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:14.668457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:14.668645 (+ 188us) server_negotiation.cc:407] Connection header received
0807 05:05:14.668842 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:14.668847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:14.668902 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:14.669011 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:14.669499 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.670045 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.670839 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.671073 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.671183 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:14.673949 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:48923: BlockingRecv error: recv got EOF from 127.1.17.194:48923 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 05:05:14 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:14 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:14.888159 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:14.882290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:58603)
0807 05:05:14.882663 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:14.882679 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:14.882828 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:14.883165 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:14.883168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:14.883455 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:14.883586 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:14.883592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.884433 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.884437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:14.885060 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:14.885072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.885512 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.885516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:14.885595 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:14.886070 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:14.886081 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:14.887980 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":101,"threads_started":1}
W20260807 05:05:14.888218 10693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:14.882327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:58603 (local address 127.1.17.254:44811)
0807 05:05:14.882462 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:14.882467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:14.882777 (+ 310us) server_negotiation.cc:407] Connection header received
0807 05:05:14.882946 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:14.882949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:14.883008 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:14.883106 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:14.883698 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.884321 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.885164 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:14.885398 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:14.885507 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:14.888110 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:58603: BlockingRecv error: recv got EOF from 127.1.17.195:58603 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 05:05:15 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:15 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:15.499045 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:15.493415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:59391)
0807 05:05:15.493682 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:15.493695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:15.493773 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:15.494013 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:15.494015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:15.494196 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:15.494323 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:15.494328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.495215 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.495221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:15.495870 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:15.495881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.496374 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.496377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:15.496468 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:15.497005 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:15.497017 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:15.498856 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
W20260807 05:05:15.499120 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:15.493476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:59391 (local address 127.1.17.254:44811)
0807 05:05:15.493716 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:15.493720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:15.493749 (+ 29us) server_negotiation.cc:407] Connection header received
0807 05:05:15.493860 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:15.493862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:15.493905 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:15.493980 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:15.494418 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.495095 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.496010 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.496253 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.496367 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:15.498984 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:59391: BlockingRecv error: recv got EOF from 127.1.17.193:59391 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":62,"threads_started":1}
Aug 07 05:05:15 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:15 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:15.680436 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:15.674842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:45143)
0807 05:05:15.675130 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:15.675162 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:15.675247 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:15.675523 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:15.675526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:15.675708 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:15.675801 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:15.675805 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.676545 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.676547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:15.677264 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:15.677275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.677746 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.677750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:15.677843 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:15.678399 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:15.678411 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:15.680311 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
W20260807 05:05:15.680583 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:15.675051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:45143 (local address 127.1.17.254:44811)
0807 05:05:15.675219 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:15.675223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:15.675236 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:15.675346 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:15.675348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:15.675390 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:15.675475 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:15.675894 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.676447 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.677383 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.677624 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.677740 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:15.680435 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:45143: BlockingRecv error: recv got EOF from 127.1.17.194:45143 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 05:05:15 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:15 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:15.894557 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:15.889105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:41303)
0807 05:05:15.889455 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:15.889468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:15.889562 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:15.889792 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:15.889795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:15.890011 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:15.890109 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:15.890113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.890830 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.890832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:15.891388 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:15.891395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.891784 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.891787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:15.891845 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:15.892348 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:15.892359 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:15.894440 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":98,"threads_started":1}
W20260807 05:05:15.894670 10697 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:15.889230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:41303 (local address 127.1.17.254:44811)
0807 05:05:15.889373 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:15.889376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:15.889534 (+ 158us) server_negotiation.cc:407] Connection header received
0807 05:05:15.889639 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:15.889641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:15.889685 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:15.889752 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:15.890210 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.890738 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.891486 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:15.891694 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:15.891767 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:15.894558 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:41303: BlockingRecv error: recv got EOF from 127.1.17.195:41303 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 05:05:16 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:16 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:16.505504 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:16.499968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:42829)
0807 05:05:16.500270 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:16.500293 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:16.500392 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:16.500650 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:16.500652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:16.500870 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:16.500976 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:16.500982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.501713 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.501716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:16.502294 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:16.502302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.502735 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.502737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:16.502805 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:16.503366 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:16.503379 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:16.505380 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
W20260807 05:05:16.505640 10701 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:16.499990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:42829 (local address 127.1.17.254:44811)
0807 05:05:16.500297 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:16.500301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:16.500343 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:05:16.500489 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:16.500491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:16.500535 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:16.500612 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:16.501072 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.501609 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.502392 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.502637 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.502714 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:16.505512 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:42829: BlockingRecv error: recv got EOF from 127.1.17.193:42829 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":119,"threads_started":1}
Aug 07 05:05:16 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:16 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:16.687143 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:16.681305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:40119)
0807 05:05:16.681605 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:16.681620 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:16.681729 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:16.682005 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:16.682008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:16.682208 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:16.682338 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:16.682343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.683311 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.683315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:16.684015 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:16.684030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.684532 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.684539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:16.684627 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:16.685115 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:16.685126 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:16.687024 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":109,"threads_started":1}
W20260807 05:05:16.687316 10701 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:16.681363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:40119 (local address 127.1.17.254:44811)
0807 05:05:16.681515 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:16.681519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:16.681695 (+ 176us) server_negotiation.cc:407] Connection header received
0807 05:05:16.681797 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:16.681799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:16.681843 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:16.681914 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:16.682461 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.683159 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.684155 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.684389 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.684514 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:16.687159 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:40119: BlockingRecv error: recv got EOF from 127.1.17.194:40119 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 05:05:16 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:16 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:16.901259 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:16.895336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:33535)
0807 05:05:16.895633 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:16.895645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:16.895791 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:16.896073 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:16.896076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:16.896290 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:16.896387 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:16.896393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.897275 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.897281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:16.897850 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:16.897858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.898333 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.898337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:16.898421 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:16.898903 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:16.898914 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:16.901140 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
W20260807 05:05:16.901484 10701 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:16.895425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:33535 (local address 127.1.17.254:44811)
0807 05:05:16.895585 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:16.895590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:16.895715 (+ 125us) server_negotiation.cc:407] Connection header received
0807 05:05:16.895890 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:16.895893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:16.895941 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:16.896048 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:16.896532 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.897164 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.897977 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:16.898221 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:16.898328 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:16.901255 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:33535: BlockingRecv error: recv got EOF from 127.1.17.195:33535 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 05:05:17 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:17 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:17.512346 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:17.506398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:36505)
0807 05:05:17.506728 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:17.506742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:17.506841 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:17.507094 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:17.507096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:17.507285 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:17.507385 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:17.507390 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.508478 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.508485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:17.509266 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:17.509277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.509895 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.509897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:17.509969 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:17.510463 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:17.510475 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:17.512231 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":94,"threads_started":1}
W20260807 05:05:17.512549 10705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:17.506452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:36505 (local address 127.1.17.254:44811)
0807 05:05:17.506687 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:17.506691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:17.506811 (+ 120us) server_negotiation.cc:407] Connection header received
0807 05:05:17.506933 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:17.506935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:17.506979 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:17.507057 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:17.507480 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.508347 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.509549 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.509793 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.509879 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:17.512368 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:36505: BlockingRecv error: recv got EOF from 127.1.17.193:36505 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
Aug 07 05:05:17 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:17 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:17.693885 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:17.687908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:55613)
0807 05:05:17.688263 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:17.688277 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:17.688374 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:17.688611 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:17.688612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:17.688882 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:17.688980 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:17.688985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.689711 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.689713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:17.690561 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:17.690572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.691181 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.691185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:17.691275 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:17.691786 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:17.691797 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:17.693717 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":137,"threads_started":1}
W20260807 05:05:17.693933 10705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:17.688013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:55613 (local address 127.1.17.254:44811)
0807 05:05:17.688185 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:17.688190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:17.688351 (+ 161us) server_negotiation.cc:407] Connection header received
0807 05:05:17.688451 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:17.688454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:17.688509 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:17.688577 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:17.689075 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.689622 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.690717 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.691041 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.691162 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:17.693832 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:55613: BlockingRecv error: recv got EOF from 127.1.17.194:55613 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 05:05:17 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:17 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:17.911818 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:17.901990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:47343)
0807 05:05:17.902330 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:17.902344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:17.902444 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:17.902739 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:17.902742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:17.902934 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:17.903051 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:17.903056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.903832 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.903835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:17.904379 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:17.904387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.904842 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.904845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:17.904924 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:17.909666 (+ 4742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:17.909678 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:17.911637 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
W20260807 05:05:17.911878 10705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:17.902078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:47343 (local address 127.1.17.254:44811)
0807 05:05:17.902239 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:17.902244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:17.902419 (+ 175us) server_negotiation.cc:407] Connection header received
0807 05:05:17.902524 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:17.902526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:17.902576 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:17.902643 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:17.903149 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.903673 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.904478 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:17.904712 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:17.904874 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:17.911767 (+ 6893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:47343: BlockingRecv error: recv got EOF from 127.1.17.195:47343 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 05:05:18 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:18 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:18.518782 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:18.513114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:45051)
0807 05:05:18.513453 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:18.513468 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:18.513568 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:18.513808 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:18.513810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:18.514032 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:18.514140 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:18.514146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.514954 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.514957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:18.515610 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:18.515618 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.516053 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.516055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:18.516119 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:18.516626 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:18.516638 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:18.518612 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":128,"threads_started":1}
W20260807 05:05:18.518855 10709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:18.513300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:45051 (local address 127.1.17.254:44811)
0807 05:05:18.513531 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:18.513535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:18.513548 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:18.513645 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:18.513647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:18.513688 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:18.513763 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:18.514264 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.514818 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.515731 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.515953 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.516029 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:18.518731 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:45051: BlockingRecv error: recv got EOF from 127.1.17.193:45051 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
Aug 07 05:05:18 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:18 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:18.700317 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:18.694762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:33529)
0807 05:05:18.695054 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:18.695067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:18.695169 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:18.695385 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:18.695388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:18.695591 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:18.695686 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:18.695690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.696568 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.696571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:18.697170 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:18.697178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.697597 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.697599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:18.697664 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:18.698243 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:18.698254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:18.700203 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
W20260807 05:05:18.700481 10709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:18.694813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:33529 (local address 127.1.17.254:44811)
0807 05:05:18.694974 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:18.694979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:18.695138 (+ 159us) server_negotiation.cc:407] Connection header received
0807 05:05:18.695236 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:18.695238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:18.695282 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:18.695347 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:18.695779 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.696445 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.697268 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.697496 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.697577 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:18.700328 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:33529: BlockingRecv error: recv got EOF from 127.1.17.194:33529 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 05:05:18 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:18 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:18.918181 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:18.912656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:39469)
0807 05:05:18.912954 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:18.912979 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:18.913092 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:18.913340 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:18.913342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:18.913551 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:18.913674 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:18.913681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.914449 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.914452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:18.915024 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:18.915033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.915437 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.915440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:18.915506 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:18.916074 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:18.916086 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:18.918040 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":136,"threads_started":1}
W20260807 05:05:18.918301 10709 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:18.912723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:39469 (local address 127.1.17.254:44811)
0807 05:05:18.912876 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:18.912880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:18.913056 (+ 176us) server_negotiation.cc:407] Connection header received
0807 05:05:18.913176 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:18.913178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:18.913224 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:18.913293 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:18.913779 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.914288 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.915131 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:18.915335 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:18.915405 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:18.918185 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:39469: BlockingRecv error: recv got EOF from 127.1.17.195:39469 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 05:05:19 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:19 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:19.525043 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:19.519608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:54023)
0807 05:05:19.519894 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:19.519909 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:19.519995 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:19.520297 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:19.520299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:19.520495 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:19.520597 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:19.520602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.521465 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.521467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:19.522001 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:19.522008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.522438 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.522440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:19.522510 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:19.522996 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:19.523007 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:19.524916 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":95,"threads_started":1}
W20260807 05:05:19.525218 10713 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:19.519659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:54023 (local address 127.1.17.254:44811)
0807 05:05:19.519928 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:19.519932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:19.519963 (+ 31us) server_negotiation.cc:407] Connection header received
0807 05:05:19.520124 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:19.520127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:19.520180 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:19.520276 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:19.520723 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.521349 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.522109 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.522334 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.522423 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:19.525055 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:54023: BlockingRecv error: recv got EOF from 127.1.17.193:54023 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":59,"threads_started":1}
Aug 07 05:05:19 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:19 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:19.706768 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:19.701035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:47265)
0807 05:05:19.701365 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:19.701378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:19.701516 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:19.701788 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:19.701791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:19.702017 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:19.702110 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:19.702116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.702842 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.702847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:19.703535 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:19.703544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.704042 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.704046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:19.704133 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:19.704701 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:19.704711 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:19.706624 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":95,"threads_started":1}
W20260807 05:05:19.706962 10713 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:19.701128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:47265 (local address 127.1.17.254:44811)
0807 05:05:19.701270 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:19.701274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:19.701449 (+ 175us) server_negotiation.cc:407] Connection header received
0807 05:05:19.701634 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:19.701636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:19.701681 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:19.701762 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:19.702221 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.702735 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.703660 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.703888 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.704026 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:19.706805 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:47265: BlockingRecv error: recv got EOF from 127.1.17.194:47265 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 05:05:19 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:19 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:19.924685 10715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:19.919039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:35847)
0807 05:05:19.919352 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:19.919369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:19.919501 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:19.919786 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:19.919788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:19.919992 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:19.920114 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:19.920120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.920927 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.920929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:19.921475 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:19.921483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.921889 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.921891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:19.921951 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:19.922432 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:19.922444 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:19.924534 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":128,"threads_started":1}
W20260807 05:05:19.924753 10713 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:19.919124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:35847 (local address 127.1.17.254:44811)
0807 05:05:19.919271 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:19.919276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:19.919442 (+ 166us) server_negotiation.cc:407] Connection header received
0807 05:05:19.919632 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:19.919634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:19.919680 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:19.919748 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:19.920213 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.920828 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.921577 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:19.921789 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:19.921857 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:19.924661 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:35847: BlockingRecv error: recv got EOF from 127.1.17.195:35847 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 05:05:20 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:20 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:20.530990 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:20.525773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:33683)
0807 05:05:20.526059 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:20.526073 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:20.526162 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:20.526417 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:20.526420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:20.526601 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:20.526726 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:20.526732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.527457 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.527459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:20.528018 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:20.528025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.528478 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.528480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:20.528541 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:20.529104 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:20.529116 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:20.530876 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":128,"threads_started":1}
W20260807 05:05:20.531144 10717 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:20.525847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:33683 (local address 127.1.17.254:44811)
0807 05:05:20.526090 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:20.526093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:20.526120 (+ 27us) server_negotiation.cc:407] Connection header received
0807 05:05:20.526254 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:20.526256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:20.526297 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:20.526377 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:20.526825 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.527358 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.528143 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.528374 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.528458 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:20.531016 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:33683: BlockingRecv error: recv got EOF from 127.1.17.193:33683 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":102,"threads_started":1}
Aug 07 05:05:20 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:20 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:20.713115 10718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:20.707559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:50161)
0807 05:05:20.707896 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:20.707909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:20.708037 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:20.708320 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:20.708323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:20.708557 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:20.708652 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:20.708658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.709443 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.709460 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:20.710010 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:20.710018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.710491 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.710495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:20.710585 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:20.711077 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:20.711087 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:20.712998 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":96,"threads_started":1}
W20260807 05:05:20.713348 10717 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:20.707657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:50161 (local address 127.1.17.254:44811)
0807 05:05:20.707821 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:20.707826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:20.708002 (+ 176us) server_negotiation.cc:407] Connection header received
0807 05:05:20.708154 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:20.708158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:20.708209 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:20.708292 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:20.708792 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.709321 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.710129 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.710356 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.710479 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:20.713134 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:50161: BlockingRecv error: recv got EOF from 127.1.17.194:50161 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 05:05:20 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:20 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:20.931129 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:20.925749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:38143)
0807 05:05:20.926044 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:20.926057 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:20.926150 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:20.926362 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:20.926364 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:20.926546 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:20.926641 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:20.926647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.927395 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.927397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:20.927963 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:20.927970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.928367 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.928369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:20.928430 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:20.929003 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:20.929014 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:20.930945 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":103,"threads_started":1}
W20260807 05:05:20.931172 10717 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:20.925875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:38143 (local address 127.1.17.254:44811)
0807 05:05:20.926023 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:20.926026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:20.926109 (+ 83us) server_negotiation.cc:407] Connection header received
0807 05:05:20.926216 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:20.926218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:20.926259 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:20.926324 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:20.926760 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.927301 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.928064 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:20.928272 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:20.928344 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:20.931062 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:38143: BlockingRecv error: recv got EOF from 127.1.17.195:38143 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 05:05:21 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:21 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:21.537726 10720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:21.531918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:55665)
0807 05:05:21.532211 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:21.532225 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:21.532321 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:21.532581 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:21.532583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:21.532815 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:21.532937 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:21.532943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.533750 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.533753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:21.534324 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:21.534332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.534749 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.534751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:21.534814 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:21.535308 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:21.535320 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:21.537547 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
W20260807 05:05:21.537801 10721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:21.531994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:55665 (local address 127.1.17.254:44811)
0807 05:05:21.532246 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:21.532249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:21.532267 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:05:21.532414 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:21.532417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:21.532464 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:21.532542 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:21.533049 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.533637 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.534432 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.534649 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.534723 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:21.537674 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:55665: BlockingRecv error: recv got EOF from 127.1.17.193:55665 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":62,"threads_started":1}
Aug 07 05:05:21 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:21 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:21.719273 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:21.713865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:37721)
0807 05:05:21.714179 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:21.714192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:21.714294 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:21.714538 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:21.714540 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:21.714725 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:21.714826 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:21.714832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.715568 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.715571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:21.716128 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:21.716136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.716596 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.716611 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:21.716689 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:21.717210 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:21.717222 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:21.719142 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":120,"threads_started":1}
W20260807 05:05:21.719439 10721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:21.713927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:37721 (local address 127.1.17.254:44811)
0807 05:05:21.714085 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:21.714090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:21.714272 (+ 182us) server_negotiation.cc:407] Connection header received
0807 05:05:21.714376 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:21.714378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:21.714436 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:21.714496 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:21.714937 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.715474 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.716235 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.716463 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.716541 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:21.719274 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:37721: BlockingRecv error: recv got EOF from 127.1.17.194:37721 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 05:05:21 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:21 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:21.937101 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:21.932101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:32979)
0807 05:05:21.932365 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:21.932377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:21.932481 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:21.932704 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:21.932706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:21.932898 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:21.932997 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:21.933002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.933710 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.933712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:21.934256 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:21.934262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.934678 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.934680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:21.934741 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:21.935211 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:21.935222 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:21.936988 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
W20260807 05:05:21.937265 10721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:21.932151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:32979 (local address 127.1.17.254:44811)
0807 05:05:21.932293 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:21.932297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:21.932466 (+ 169us) server_negotiation.cc:407] Connection header received
0807 05:05:21.932554 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:21.932556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:21.932602 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:21.932663 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:21.933091 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.933627 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.934355 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:21.934579 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:21.934657 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:21.937112 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:32979: BlockingRecv error: recv got EOF from 127.1.17.195:32979 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 05:05:22 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:22 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:22.544111 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:22.538640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:42773)
0807 05:05:22.538932 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:22.538946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:22.539052 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:22.539308 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:22.539310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:22.539495 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:22.539609 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:22.539614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.540338 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.540342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:22.541024 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:22.541034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.541466 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.541469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:22.541534 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:22.542054 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:22.542065 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:22.543994 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
W20260807 05:05:22.544318 10725 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:22.538723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:42773 (local address 127.1.17.254:44811)
0807 05:05:22.538956 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:22.538960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:22.538992 (+ 32us) server_negotiation.cc:407] Connection header received
0807 05:05:22.539131 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:22.539133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:22.539194 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:22.539271 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:22.539699 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.540215 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.541138 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.541365 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.541440 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:22.544186 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:42773: BlockingRecv error: recv got EOF from 127.1.17.193:42773 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
Aug 07 05:05:22 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:22 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:22.725590 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:22.720016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:52017)
0807 05:05:22.720281 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:22.720294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:22.720393 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:22.720677 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:22.720679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:22.720908 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:22.721011 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:22.721017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.721783 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.721788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:22.722360 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:22.722368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.722831 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.722834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:22.722924 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:22.723473 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:22.723483 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:22.725473 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
W20260807 05:05:22.725733 10725 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:22.720080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:52017 (local address 127.1.17.254:44811)
0807 05:05:22.720244 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:22.720252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:22.720359 (+ 107us) server_negotiation.cc:407] Connection header received
0807 05:05:22.720469 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:22.720471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:22.720515 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:22.720636 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:22.721133 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.721658 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.722485 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.722719 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.722823 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:22.725619 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:52017: BlockingRecv error: recv got EOF from 127.1.17.194:52017 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 05:05:22 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:22 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:22.950995 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:22.937822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:41311)
0807 05:05:22.938147 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:22.938166 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:22.938303 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:22.938579 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:22.938584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:22.938855 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:22.939025 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:22.939036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.939794 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.939800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:22.941014 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:22.941030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.941474 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.941479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:22.941603 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:22.948584 (+ 6981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:22.948604 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:22.950831 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
W20260807 05:05:22.951102 10725 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:22.937954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:41311 (local address 127.1.17.254:44811)
0807 05:05:22.938096 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:22.938100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:22.938246 (+ 146us) server_negotiation.cc:407] Connection header received
0807 05:05:22.938400 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:22.938403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:22.938451 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:22.938519 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:22.939164 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.939685 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.941138 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:22.941348 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:22.941427 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:22.950989 (+ 9562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:41311: BlockingRecv error: recv got EOF from 127.1.17.195:41311 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 05:05:23 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:23 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:23.551112 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:23.545083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:55143)
0807 05:05:23.545377 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:23.545391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:23.545500 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:23.545715 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:23.545717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:23.545905 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:23.546010 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:23.546015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.546827 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.546830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:23.547429 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:23.547439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.547990 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.547993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:23.548068 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:23.548687 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:23.548700 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:23.550907 (+ 2207us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":106,"threads_started":1}
W20260807 05:05:23.551368 10729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:23.545162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:55143 (local address 127.1.17.254:44811)
0807 05:05:23.545401 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:23.545405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:23.545453 (+ 48us) server_negotiation.cc:407] Connection header received
0807 05:05:23.545566 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:23.545567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:23.545609 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:23.545683 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:23.546166 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.546710 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.547556 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.547785 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.547866 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:23.551207 (+ 3341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:55143: BlockingRecv error: recv got EOF from 127.1.17.193:55143 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
Aug 07 05:05:23 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:23 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:23.732004 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:23.726382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:56265)
0807 05:05:23.726724 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:23.726740 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:23.726897 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:23.727198 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:23.727201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:23.727440 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:23.727536 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:23.727542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.728314 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.728329 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:23.728967 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:23.728976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.729446 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.729450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:23.729541 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:23.730045 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:23.730056 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:23.731844 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":119,"threads_started":1}
W20260807 05:05:23.732084 10729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:23.726538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:56265 (local address 127.1.17.254:44811)
0807 05:05:23.726719 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:23.726724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:23.726839 (+ 115us) server_negotiation.cc:407] Connection header received
0807 05:05:23.727016 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:23.727021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:23.727076 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:23.727167 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:23.727659 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.728201 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.729092 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.729326 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.729447 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:23.731984 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:56265: BlockingRecv error: recv got EOF from 127.1.17.194:56265 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 05:05:23 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:23 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:23.957428 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:23.951906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:42369)
0807 05:05:23.952211 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:23.952224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:23.952338 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:23.952573 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:23.952575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:23.952764 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:23.952891 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:23.952896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.953654 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.953657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:23.954222 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:23.954230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.954673 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.954676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:23.954766 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:23.955275 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:23.955286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:23.957273 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
W20260807 05:05:23.957576 10729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:23.951971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:42369 (local address 127.1.17.254:44811)
0807 05:05:23.952099 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:23.952106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:23.952297 (+ 191us) server_negotiation.cc:407] Connection header received
0807 05:05:23.952407 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:23.952408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:23.952459 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:23.952535 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:23.953002 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.953545 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.954327 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:23.954541 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:23.954634 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:23.957420 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:42369: BlockingRecv error: recv got EOF from 127.1.17.195:42369 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 05:05:24 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:24 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:24.557632 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:24.551953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:58551)
0807 05:05:24.552246 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:24.552264 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:24.552375 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:24.552600 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:24.552602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:24.552866 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:24.553014 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:24.553025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.553771 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.553773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:24.554360 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:24.554368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.554793 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.554795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:24.554855 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:24.555393 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:24.555405 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:24.557446 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":75,"threads_started":1}
W20260807 05:05:24.557693 10733 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:24.552050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:58551 (local address 127.1.17.254:44811)
0807 05:05:24.552381 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:24.552385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:24.552398 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:05:24.552451 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:24.552453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:24.552493 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:24.552563 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:24.553130 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.553675 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.554477 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.554699 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.554774 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:24.557565 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:58551: BlockingRecv error: recv got EOF from 127.1.17.193:58551 (error 108)
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":77,"threads_started":1}
Aug 07 05:05:24 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:24 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:24.738637 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:24.732972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:46519)
0807 05:05:24.733247 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:24.733261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:24.733396 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:24.733679 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:24.733681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:24.733897 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:24.733989 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:24.733995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.734725 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.734730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:24.735325 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:24.735332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.735786 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.735789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:24.735875 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:24.736404 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:24.736417 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:24.738443 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":85,"threads_started":1}
W20260807 05:05:24.738665 10733 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:24.733027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:46519 (local address 127.1.17.254:44811)
0807 05:05:24.733198 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:24.733203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:24.733345 (+ 142us) server_negotiation.cc:407] Connection header received
0807 05:05:24.733512 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:24.733516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:24.733567 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:24.733652 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:24.734109 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.734616 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.735445 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.735676 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.735782 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:24.738562 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:46519: BlockingRecv error: recv got EOF from 127.1.17.194:46519 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 05:05:24 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:24 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.195@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:24.964493 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:24.958384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.195:59839)
0807 05:05:24.958664 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:24.958677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:24.958786 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:24.959050 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:24.959052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:24.959236 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:24.959334 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:24.959340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.960225 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.960229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:24.961191 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:24.961204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.961819 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.961822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:24.961929 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:24.962450 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:24.962463 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:24.964373 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":111,"threads_started":1}
W20260807 05:05:24.964655 10733 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:24.958481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:59839 (local address 127.1.17.254:44811)
0807 05:05:24.958613 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:24.958616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:24.958754 (+ 138us) server_negotiation.cc:407] Connection header received
0807 05:05:24.958859 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:24.958861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:24.958928 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:24.958993 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:24.959429 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.960060 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.961346 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:24.961692 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:24.961780 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:24.964506 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.195:59839: BlockingRecv error: recv got EOF from 127.1.17.195:59839 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 05:05:25 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:25 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.193@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:25.564267 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:25.558598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.193:51827)
0807 05:05:25.558888 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:25.558900 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:25.558990 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:25.559227 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:25.559229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:25.559412 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:25.559526 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:25.559531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.560235 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.560238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:25.560831 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:25.560839 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.561392 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.561395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:25.561466 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:25.561968 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:25.561980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:25.564148 (+ 2168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
W20260807 05:05:25.564440 10737 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:25.558661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:51827 (local address 127.1.17.254:44811)
0807 05:05:25.558912 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:25.558915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:25.558976 (+ 61us) server_negotiation.cc:407] Connection header received
0807 05:05:25.559068 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:25.559070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:25.559114 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:25.559189 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:25.559619 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.560141 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.560945 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.561259 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.561380 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:25.564276 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.193:51827: BlockingRecv error: recv got EOF from 127.1.17.193:51827 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":68,"threads_started":1}
Aug 07 05:05:25 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 05:05:25 dist-test-slave-1x4g krb5kdc[9617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.17.194@KRBTEST.COM for oryx/127.1.17.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 05:05:25.745538 10738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:25.739635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44811 (local address 127.1.17.194:37969)
0807 05:05:25.739922 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:25.739935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:25.740034 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:25.740351 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:25.740353 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:25.740545 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:25.740639 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:25.740644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.741454 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.741457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:25.742033 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:25.742040 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.742498 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.742500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:25.742560 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:25.743120 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:25.743131 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:25.745322 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.17.254:44811: Server oryx/127.1.17.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
W20260807 05:05:25.745591 10737 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:25.739820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:37969 (local address 127.1.17.254:44811)
0807 05:05:25.739995 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:25.739999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:25.740013 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:05:25.740176 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:25.740179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:25.740231 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:25.740314 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:25.740748 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.741349 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.742141 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:25.742379 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:25.742481 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:25.745456 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.17.194:37969: BlockingRecv error: recv got EOF from 127.1.17.194:37969 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:05:25.846099 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10240
I20260807 05:05:25.854525 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10399
I20260807 05:05:25.861578 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10536
I20260807 05:05:25.867828 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10168
2026-08-07T05:05:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18001 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T05:05:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:05:25Z Disabled control of system clock
I20260807 05:05:25.900527 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:45497
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:45497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 05:05:26.013293 10742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:26.013542 10742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:26.013576 10742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:26.017616 10742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:26.017666 10742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:26.017683 10742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:26.017699 10742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:26.017715 10742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:26.022310 10742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:45497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:45497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:26.023304 10742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:26.024626 10742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:26.033108 10749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.033190 10742 server_base.cc:1034] running on GCE node
W20260807 05:05:26.033138 10751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:26.033147 10748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.033969 10742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:26.035024 10742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:26.036214 10742 hybrid_clock.cc:648] HybridClock initialized: now 1786079126036185 us; error 41 us; skew 500 ppm
I20260807 05:05:26.038115 10742 webserver.cc:460] Webserver started at http://127.1.17.254:40959/ using document root <none> and password file <none>
I20260807 05:05:26.038687 10742 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:26.038776 10742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:26.039003 10742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:26.041082 10742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "6f101c6f2ce740199d45da1a1b8398a3"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "3e2159203c41c8b297bf131298986b50"
server_key_iv: "0cbe1d7d59b7eda61067b23f301280b3"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.041641 10742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "6f101c6f2ce740199d45da1a1b8398a3"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "3e2159203c41c8b297bf131298986b50"
server_key_iv: "0cbe1d7d59b7eda61067b23f301280b3"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.045185 10742 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 05:05:26.047509 10758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.048662 10742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:26.048823 10742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "6f101c6f2ce740199d45da1a1b8398a3"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "3e2159203c41c8b297bf131298986b50"
server_key_iv: "0cbe1d7d59b7eda61067b23f301280b3"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.048964 10742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:26.059214 10742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:26.059811 10742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:26.060014 10742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:26.066756 10742 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:45497
I20260807 05:05:26.066766 10811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:45497 every 8 connection(s)
I20260807 05:05:26.067634 10742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:05:26.070418 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:26.075573 10812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3: Bootstrap starting.
I20260807 05:05:26.076048 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10742
I20260807 05:05:26.076189 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:05:26.076483 1095 external_mini_cluster.cc:1428] Setting key 140b730a166be298bd953938b2b2417a
I20260807 05:05:26.078421 10812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:26.079288 10812 log.cc:824] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:26.081494 10812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3: No bootstrap required, opened a new log
I20260807 05:05:26.085727 10812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f101c6f2ce740199d45da1a1b8398a3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45497 } }
I20260807 05:05:26.085981 10812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:26.086040 10812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f101c6f2ce740199d45da1a1b8398a3, State: Initialized, Role: FOLLOWER
I20260807 05:05:26.086496 10812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f101c6f2ce740199d45da1a1b8398a3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45497 } }
I20260807 05:05:26.086655 10812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:05:26.086722 10812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:05:26.086815 10812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:26.086833 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.077992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.1.17.254:45497)
0807 05:05:26.078665 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:26.078687 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:26.078733 (+ 46us) server_negotiation.cc:407] Connection header received
0807 05:05:26.079537 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:26.079560 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:26.079941 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:26.080295 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:26.080697 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.082188 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.083101 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.083791 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.083968 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:26.084766 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:26.084799 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:26.084817 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:26.085318 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:26.085386 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:26.085410 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:26.085501 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:26.085653 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 05:05:26.085907 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260807 05:05:26.087726 10812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f101c6f2ce740199d45da1a1b8398a3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45497 } }
I20260807 05:05:26.088079 10812 leader_election.cc:302] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f101c6f2ce740199d45da1a1b8398a3; no voters:
I20260807 05:05:26.088395 10812 leader_election.cc:288] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:05:26.088511 10818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:26.088795 10818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [term 1 LEADER]: Becoming Leader. State: Replica: 6f101c6f2ce740199d45da1a1b8398a3, State: Running, Role: LEADER
I20260807 05:05:26.089403 10818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f101c6f2ce740199d45da1a1b8398a3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45497 } }
I20260807 05:05:26.089628 10812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:26.090967 10820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f101c6f2ce740199d45da1a1b8398a3. Latest consensus state: current_term: 1 leader_uuid: "6f101c6f2ce740199d45da1a1b8398a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f101c6f2ce740199d45da1a1b8398a3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45497 } } }
I20260807 05:05:26.091094 10820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:26.091316 10819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f101c6f2ce740199d45da1a1b8398a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f101c6f2ce740199d45da1a1b8398a3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 45497 } } }
I20260807 05:05:26.091481 10819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:26.091548 10828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:05:26.094408 10828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:05:26.099442 10828 catalog_manager.cc:1347] Generated new cluster ID: 58617909d2ac4a8eaab1b19e0aa0cb67
I20260807 05:05:26.099504 10828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:05:26.110888 10828 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:05:26.111796 10828 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:05:26.117993 10828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f101c6f2ce740199d45da1a1b8398a3: Generated new TSK 0
I20260807 05:05:26.118707 10828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:05:26.126425 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45497
--builtin_ntp_servers=127.1.17.212:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 05:05:26.241111 10837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:26.241389 10837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:26.241472 10837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:26.245517 10837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:26.245621 10837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:26.245703 10837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:05:26.250414 10837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:26.251490 10837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:26.252919 10837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:26.262593 10845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:26.262602 10843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.262740 10837 server_base.cc:1034] running on GCE node
W20260807 05:05:26.262643 10842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.263414 10837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:26.264048 10837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:26.265225 10837 hybrid_clock.cc:648] HybridClock initialized: now 1786079126265204 us; error 38 us; skew 500 ppm
I20260807 05:05:26.267079 10837 webserver.cc:460] Webserver started at http://127.1.17.193:41553/ using document root <none> and password file <none>
I20260807 05:05:26.267668 10837 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:26.267733 10837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:26.267975 10837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:26.270109 10837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "b4bd822db6ec465c86666814ff410b4a"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "b711b399bb4488272a5bd11a676f9d50"
server_key_iv: "2ebfe3a8b8ab4eb4cd50df77326591f1"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.270684 10837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "b4bd822db6ec465c86666814ff410b4a"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "b711b399bb4488272a5bd11a676f9d50"
server_key_iv: "2ebfe3a8b8ab4eb4cd50df77326591f1"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.274507 10837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:05:26.277267 10851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.278497 10837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:26.278635 10837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "b4bd822db6ec465c86666814ff410b4a"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "b711b399bb4488272a5bd11a676f9d50"
server_key_iv: "2ebfe3a8b8ab4eb4cd50df77326591f1"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.278753 10837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:26.294656 10837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:26.295325 10837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:26.295524 10837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:26.296103 10837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:26.297111 10837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:26.297183 10837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.297250 10837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:26.297299 10837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.306571 10837 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:45003
I20260807 05:05:26.306603 10967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:45003 every 8 connection(s)
I20260807 05:05:26.307505 10837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:05:26.313987 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10837
I20260807 05:05:26.314106 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:05:26.314431 1095 external_mini_cluster.cc:1428] Setting key 9d3b99b3916ea20d0071fb304d45b77a
I20260807 05:05:26.316730 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45497
--builtin_ntp_servers=127.1.17.212:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 05:05:26.317606 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.309358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:57067 (local address 127.1.17.254:45497)
0807 05:05:26.309502 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:26.309506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:26.310592 (+ 1086us) server_negotiation.cc:407] Connection header received
0807 05:05:26.312032 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:26.312035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:26.312082 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:26.312159 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:26.313140 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.313681 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.315339 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.315677 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.315793 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:26.317141 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:26.317144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:26.317146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:26.317160 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:26.317179 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:26.317187 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:26.317253 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:26.317449 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 05:05:26.317499 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:05:26.318822 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.309762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45497 (local address 127.1.17.193:57067)
0807 05:05:26.310446 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:26.310487 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:26.311808 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:26.312334 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:26.312341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:26.312426 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:26.313009 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:26.313026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.313814 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.313818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:26.315172 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:26.315185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.315847 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.315852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:26.316074 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:26.316732 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:05:26.316734 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:26.316822 (+ 88us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:05:26.316915 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:26.317279 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:26.317292 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:26.317301 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 05:05:26.317435 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 05:05:26.317747 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":119,"threads_started":1}
I20260807 05:05:26.320024 10969 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45497
I20260807 05:05:26.320279 10969 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:26.320866 10969 heartbeater.cc:511] Master 127.1.17.254:45497 requested a full tablet report, sending...
I20260807 05:05:26.322384 10775 ts_manager.cc:194] Registered new tserver with Master: b4bd822db6ec465c86666814ff410b4a (127.1.17.193:45003)
I20260807 05:05:26.323535 10775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.193:57067
W20260807 05:05:26.432330 10972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:26.432611 10972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:26.432677 10972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:26.436687 10972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:26.436765 10972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:26.436869 10972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:05:26.441628 10972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:26.442696 10972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:26.444083 10972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:26.453518 10977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:26.453507 10978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:26.453596 10980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.453754 10972 server_base.cc:1034] running on GCE node
I20260807 05:05:26.454309 10972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:26.454957 10972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:26.456152 10972 hybrid_clock.cc:648] HybridClock initialized: now 1786079126456116 us; error 46 us; skew 500 ppm
I20260807 05:05:26.458086 10972 webserver.cc:460] Webserver started at http://127.1.17.194:41249/ using document root <none> and password file <none>
I20260807 05:05:26.458683 10972 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:26.458773 10972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:26.459008 10972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:26.461169 10972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "70c807b8c7c241cb9b8178a555e743b9"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "4a632f50fb7dfd00944cfcf63e4a9272"
server_key_iv: "c04393468b21c9a6fea1ee20bd0d53d3"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.461748 10972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "70c807b8c7c241cb9b8178a555e743b9"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "4a632f50fb7dfd00944cfcf63e4a9272"
server_key_iv: "c04393468b21c9a6fea1ee20bd0d53d3"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.465584 10972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:05:26.468144 10986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.469421 10972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:26.469564 10972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "70c807b8c7c241cb9b8178a555e743b9"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "4a632f50fb7dfd00944cfcf63e4a9272"
server_key_iv: "c04393468b21c9a6fea1ee20bd0d53d3"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.469681 10972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:26.486020 10972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:26.486704 10972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:26.486902 10972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:26.487481 10972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:26.488402 10972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:26.488473 10972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.488545 10972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:26.488588 10972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.497771 10972 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:39479
I20260807 05:05:26.497809 11103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:39479 every 8 connection(s)
I20260807 05:05:26.498643 10972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:05:26.504551 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 10972
I20260807 05:05:26.504678 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:05:26.504981 1095 external_mini_cluster.cc:1428] Setting key 6049057ad157d72abe66d6dc1460b858
I20260807 05:05:26.506979 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45497
--builtin_ntp_servers=127.1.17.212:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 05:05:26.508227 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.500334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:40219 (local address 127.1.17.254:45497)
0807 05:05:26.500468 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:26.500472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:26.501534 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 05:05:26.502869 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:26.502872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:26.502928 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:26.503005 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:26.503915 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.504573 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.506055 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.506318 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.506403 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:26.507546 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:26.507553 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:26.507559 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:26.507577 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:26.507605 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:26.507619 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:26.507701 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:26.507935 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 05:05:26.508009 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:05:26.509066 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.500676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45497 (local address 127.1.17.194:40219)
0807 05:05:26.501386 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:26.501425 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:26.502672 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:26.503170 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:26.503177 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:26.503259 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:26.503762 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:26.503774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.504754 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.504758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:26.505923 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:26.505934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.506490 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.506494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:26.506633 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:26.507275 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:05:26.507276 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:26.507316 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:05:26.507379 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:26.507768 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:26.507778 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:26.507783 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 05:05:26.507901 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 05:05:26.508122 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":133,"threads_started":1}
I20260807 05:05:26.510234 11104 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45497
I20260807 05:05:26.510491 11104 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:26.511003 11104 heartbeater.cc:511] Master 127.1.17.254:45497 requested a full tablet report, sending...
I20260807 05:05:26.512090 10775 ts_manager.cc:194] Registered new tserver with Master: 70c807b8c7c241cb9b8178a555e743b9 (127.1.17.194:39479)
I20260807 05:05:26.512998 10775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.194:40219
W20260807 05:05:26.621539 11107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:26.621767 11107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:26.621800 11107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:26.625726 11107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:26.625771 11107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:26.625818 11107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:05:26.630666 11107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:45497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:26.631651 11107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:26.633074 11107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:26.642745 11113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:26.642704 11114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:26.642695 11116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.642863 11107 server_base.cc:1034] running on GCE node
I20260807 05:05:26.643424 11107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:26.644104 11107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:26.645321 11107 hybrid_clock.cc:648] HybridClock initialized: now 1786079126645304 us; error 34 us; skew 500 ppm
I20260807 05:05:26.647135 11107 webserver.cc:460] Webserver started at http://127.1.17.195:34623/ using document root <none> and password file <none>
I20260807 05:05:26.647715 11107 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:26.647807 11107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:26.648041 11107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:26.650100 11107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "6fc0c1aac4464bcab846579f86e69221"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "c9010c5c819dc29560307f8b0337b61e"
server_key_iv: "ce3ff075460cab80e26766d0fd54cd47"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.650647 11107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "6fc0c1aac4464bcab846579f86e69221"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "c9010c5c819dc29560307f8b0337b61e"
server_key_iv: "ce3ff075460cab80e26766d0fd54cd47"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.654304 11107 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 05:05:26.656672 11122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.657966 11107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:05:26.658097 11107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "6fc0c1aac4464bcab846579f86e69221"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "c9010c5c819dc29560307f8b0337b61e"
server_key_iv: "ce3ff075460cab80e26766d0fd54cd47"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.658206 11107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:26.678608 11107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:26.679293 11107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:26.679493 11107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:26.680058 11107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:26.681049 11107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:26.681116 11107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.681185 11107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:26.681233 11107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.690380 11107 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:34805
I20260807 05:05:26.690490 11239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:34805 every 8 connection(s)
I20260807 05:05:26.691428 11107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:05:26.694468 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 11107
I20260807 05:05:26.694576 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:05:26.694832 1095 external_mini_cluster.cc:1428] Setting key e32b2676abb7e8bf4a1a55a1291d9c34
I20260807 05:05:26.700855 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.693305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:58823 (local address 127.1.17.254:45497)
0807 05:05:26.693459 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:26.693463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:26.694531 (+ 1068us) server_negotiation.cc:407] Connection header received
0807 05:05:26.696098 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:26.696102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:26.696158 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:26.696265 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:26.697283 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.697816 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.698954 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.699187 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.699297 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:26.700336 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:26.700340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:26.700342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:26.700361 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:26.700380 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:26.700388 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:26.700476 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:26.700634 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 05:05:26.700693 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:05:26.701704 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.693709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:45497 (local address 127.1.17.195:58823)
0807 05:05:26.694366 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:26.694416 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:26.695886 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:26.696442 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:26.696449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:26.696529 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:26.697141 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:26.697156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.697962 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.697965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:26.698828 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:26.698837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:26.699338 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:26.699342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:26.699507 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:26.700088 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:05:26.700089 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:26.700128 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:05:26.700189 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:26.700476 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:26.700488 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:26.700497 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 05:05:26.700631 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 05:05:26.700923 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":122,"threads_started":1}
I20260807 05:05:26.702739 11240 heartbeater.cc:347] Connected to a master server at 127.1.17.254:45497
I20260807 05:05:26.703028 11240 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:26.703540 11240 heartbeater.cc:511] Master 127.1.17.254:45497 requested a full tablet report, sending...
I20260807 05:05:26.704619 10775 ts_manager.cc:194] Registered new tserver with Master: 6fc0c1aac4464bcab846579f86e69221 (127.1.17.195:34805)
I20260807 05:05:26.705274 10775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.195:58823
I20260807 05:05:26.708606 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 05:05:26.711459 10816 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:26.710539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.1.17.254:45497)
0807 05:05:26.710657 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:26.710660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:26.710885 (+ 225us) server_negotiation.cc:407] Connection header received
0807 05:05:26.710979 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:26.710981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:26.711015 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:26.711080 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:26.711338 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33610: BlockingRecv error: recv got EOF from 127.0.0.1:33610 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:05:26.711699 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10837
I20260807 05:05:26.721601 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10972
I20260807 05:05:26.728364 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 11107
I20260807 05:05:26.734504 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 10742
2026-08-07T05:05:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (865 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T05:05:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:05:26Z Disabled control of system clock
I20260807 05:05:26.765974 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44023
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:44023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 05:05:26.880542 11251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:26.880832 11251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:26.880897 11251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:26.884670 11251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:26.884716 11251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:26.884732 11251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:26.884747 11251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:26.884761 11251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:26.889230 11251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:44023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:26.890118 11251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:26.891383 11251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:26.899935 11257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.899993 11251 server_base.cc:1034] running on GCE node
W20260807 05:05:26.899951 11259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:26.899943 11256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:26.900753 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:26.901798 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:26.902968 11251 hybrid_clock.cc:648] HybridClock initialized: now 1786079126902950 us; error 30 us; skew 500 ppm
I20260807 05:05:26.904678 11251 webserver.cc:460] Webserver started at http://127.1.17.254:40677/ using document root <none> and password file <none>
I20260807 05:05:26.905256 11251 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:26.905328 11251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:26.905484 11251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:26.907491 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "632f940d9b3641378f4fd114702745e5"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "d806e77931cf129b3aef230c3b05bc86"
server_key_iv: "9daf7842dae46d2de1ec91179e5f6b4f"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.908004 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "632f940d9b3641378f4fd114702745e5"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "d806e77931cf129b3aef230c3b05bc86"
server_key_iv: "9daf7842dae46d2de1ec91179e5f6b4f"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.911505 11251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:05:26.913793 11265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:26.914981 11251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:26.915102 11251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "632f940d9b3641378f4fd114702745e5"
format_stamp: "Formatted at 2026-08-07 05:05:26 on dist-test-slave-1x4g"
server_key: "d806e77931cf129b3aef230c3b05bc86"
server_key_iv: "9daf7842dae46d2de1ec91179e5f6b4f"
server_key_version: "encryptionkey@0"
I20260807 05:05:26.915210 11251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:26.919715 11251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:26.920285 11251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:26.920466 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:26.927132 11318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:44023 every 8 connection(s)
I20260807 05:05:26.927147 11251 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:44023
I20260807 05:05:26.927997 11251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:05:26.931093 11319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:26.936291 11319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5: Bootstrap starting.
I20260807 05:05:26.937476 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 11251
I20260807 05:05:26.937565 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:05:26.937856 1095 external_mini_cluster.cc:1428] Setting key f22ccd531be538b110c50926112f96ac
I20260807 05:05:26.938879 11319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:26.939721 11319 log.cc:824] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:26.941824 11319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5: No bootstrap required, opened a new log
I20260807 05:05:26.943142 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:26.938996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60274 (local address 127.1.17.254:44023)
0807 05:05:26.939650 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:26.939664 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:26.939698 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:05:26.940502 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:26.940526 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:26.940870 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:26.941181 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:26.941949 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:26.941955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:26.941972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:26.941985 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:26.942046 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:26.942062 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:26.942177 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:26.942282 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 05:05:26.942533 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":132,"threads_started":1}
I20260807 05:05:26.945844 11319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632f940d9b3641378f4fd114702745e5" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44023 } }
I20260807 05:05:26.946069 11319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:26.946146 11319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632f940d9b3641378f4fd114702745e5, State: Initialized, Role: FOLLOWER
I20260807 05:05:26.946585 11319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632f940d9b3641378f4fd114702745e5" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44023 } }
I20260807 05:05:26.946732 11319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:05:26.946817 11319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:05:26.946916 11319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:26.947819 11319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632f940d9b3641378f4fd114702745e5" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44023 } }
I20260807 05:05:26.948156 11319 leader_election.cc:302] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 632f940d9b3641378f4fd114702745e5; no voters:
I20260807 05:05:26.948472 11319 leader_election.cc:288] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:05:26.948606 11324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:26.948846 11324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [term 1 LEADER]: Becoming Leader. State: Replica: 632f940d9b3641378f4fd114702745e5, State: Running, Role: LEADER
I20260807 05:05:26.949200 11324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632f940d9b3641378f4fd114702745e5" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44023 } }
I20260807 05:05:26.949705 11319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:26.951007 11326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 632f940d9b3641378f4fd114702745e5. Latest consensus state: current_term: 1 leader_uuid: "632f940d9b3641378f4fd114702745e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632f940d9b3641378f4fd114702745e5" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44023 } } }
I20260807 05:05:26.951022 11325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "632f940d9b3641378f4fd114702745e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632f940d9b3641378f4fd114702745e5" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44023 } } }
I20260807 05:05:26.951151 11326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:26.951151 11325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5 [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:26.951642 11334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:05:26.954721 11334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:05:26.959699 11334 catalog_manager.cc:1347] Generated new cluster ID: 4c6a8d30658145c99cec335d717ea0ec
I20260807 05:05:26.959764 11334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:05:26.974884 11334 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:05:26.975234 11334 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:05:26.984122 11334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 632f940d9b3641378f4fd114702745e5: Generated new TSK 0
I20260807 05:05:26.984865 11334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:05:26.995922 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44023
--builtin_ntp_servers=127.1.17.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 05:05:27.112952 11344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:27.113265 11344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:27.113355 11344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:27.117334 11344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:27.117411 11344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:27.117501 11344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:05:27.122244 11344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:27.123248 11344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:27.124648 11344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:27.134318 11350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:27.134310 11349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:27.134529 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:27.134564 11344 server_base.cc:1034] running on GCE node
I20260807 05:05:27.135016 11344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:27.135555 11344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:27.136718 11344 hybrid_clock.cc:648] HybridClock initialized: now 1786079127136696 us; error 32 us; skew 500 ppm
I20260807 05:05:27.138470 11344 webserver.cc:460] Webserver started at http://127.1.17.193:38959/ using document root <none> and password file <none>
I20260807 05:05:27.139040 11344 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:27.139119 11344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:27.139362 11344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:27.141449 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "0feb135e920b48b69acc7ebd2c208f69"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "a7d898fcdc7933a0066db56b0928e889"
server_key_iv: "0c99cd69048d8a97968b7ae650fdc250"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.142045 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "0feb135e920b48b69acc7ebd2c208f69"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "a7d898fcdc7933a0066db56b0928e889"
server_key_iv: "0c99cd69048d8a97968b7ae650fdc250"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.145663 11344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:05:27.148056 11358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.149300 11344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:05:27.149428 11344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "0feb135e920b48b69acc7ebd2c208f69"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "a7d898fcdc7933a0066db56b0928e889"
server_key_iv: "0c99cd69048d8a97968b7ae650fdc250"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.149538 11344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:27.154245 11344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:27.154824 11344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:27.155011 11344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:27.155534 11344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:27.156422 11344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:27.156495 11344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.156564 11344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:27.156626 11344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.165674 11344 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:40445
I20260807 05:05:27.165747 11477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:40445 every 8 connection(s)
I20260807 05:05:27.166514 11344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:05:27.172343 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:27.168227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:45165 (local address 127.1.17.254:44023)
0807 05:05:27.168370 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:27.168374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:27.169422 (+ 1048us) server_negotiation.cc:407] Connection header received
0807 05:05:27.170723 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:27.170726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:27.170772 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:27.170854 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:27.171861 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:27.171864 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:27.171867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:27.171879 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:27.171902 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:27.171918 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:27.172013 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:27.172166 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 05:05:27.172212 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:05:27.173055 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 11344
I20260807 05:05:27.173169 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:05:27.173341 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:27.168532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44023 (local address 127.1.17.193:45165)
0807 05:05:27.169266 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:27.169305 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:27.170506 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:27.171018 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:27.171025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:27.171110 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:27.171610 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:05:27.171612 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:27.171651 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:05:27.171716 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:27.172018 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:27.172031 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:27.172037 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:05:27.172207 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 05:05:27.172438 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":121,"threads_started":1}
I20260807 05:05:27.173512 1095 external_mini_cluster.cc:1428] Setting key 8df2b2d6f653198a2c479f412302c2a3
I20260807 05:05:27.174562 11478 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44023
I20260807 05:05:27.174787 11478 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:27.175215 11478 heartbeater.cc:511] Master 127.1.17.254:44023 requested a full tablet report, sending...
I20260807 05:05:27.175578 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44023
--builtin_ntp_servers=127.1.17.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 05:05:27.176900 11282 ts_manager.cc:194] Registered new tserver with Master: 0feb135e920b48b69acc7ebd2c208f69 (127.1.17.193:40445)
W20260807 05:05:27.289726 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:27.290012 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:27.290086 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:27.293943 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:27.294024 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:27.294113 11481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:05:27.298918 11481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:27.299933 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:27.301307 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:27.310799 11489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:27.310806 11486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:27.310814 11487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:27.310866 11481 server_base.cc:1034] running on GCE node
I20260807 05:05:27.311484 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:27.312016 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:27.313207 11481 hybrid_clock.cc:648] HybridClock initialized: now 1786079127313188 us; error 28 us; skew 500 ppm
I20260807 05:05:27.314990 11481 webserver.cc:460] Webserver started at http://127.1.17.194:34045/ using document root <none> and password file <none>
I20260807 05:05:27.315510 11481 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:27.315567 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:27.315728 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:27.317800 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "591c63cfda704584b3f81c0612fb5552"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "425f96a3d289980717810711324aaa25"
server_key_iv: "85c95d5349bed70f3a50ea81742b5dc4"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.318347 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "591c63cfda704584b3f81c0612fb5552"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "425f96a3d289980717810711324aaa25"
server_key_iv: "85c95d5349bed70f3a50ea81742b5dc4"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.322021 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:05:27.324379 11495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.325671 11481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:27.325765 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "591c63cfda704584b3f81c0612fb5552"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "425f96a3d289980717810711324aaa25"
server_key_iv: "85c95d5349bed70f3a50ea81742b5dc4"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.325850 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:27.330256 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:27.330839 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:27.331024 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:27.331557 11481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:27.332463 11481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:27.332533 11481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.332597 11481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:27.332644 11481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.341423 11481 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:35957
I20260807 05:05:27.341435 11616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:35957 every 8 connection(s)
I20260807 05:05:27.342242 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:05:27.343317 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 11481
I20260807 05:05:27.343436 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:05:27.343743 1095 external_mini_cluster.cc:1428] Setting key 6875bc89f8a3b22d3dab2d3b1860800f
I20260807 05:05:27.345885 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44023
--builtin_ntp_servers=127.1.17.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 05:05:27.348856 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:27.344171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57515 (local address 127.1.17.254:44023)
0807 05:05:27.344370 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:27.344375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:27.345760 (+ 1385us) server_negotiation.cc:407] Connection header received
0807 05:05:27.347076 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:27.347082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:27.347133 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:27.347229 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:27.348339 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:27.348344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:27.348348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:27.348363 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:27.348390 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:27.348402 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:27.348511 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:27.348608 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 05:05:27.348653 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 05:05:27.349830 11619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:27.344648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44023 (local address 127.1.17.194:57515)
0807 05:05:27.345585 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:27.345661 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:27.346870 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:27.347376 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:27.347383 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:27.347477 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:27.348082 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:05:27.348084 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:27.348123 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:05:27.348190 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:27.348491 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:27.348496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:27.348500 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:05:27.348618 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 05:05:27.348875 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":192,"threads_started":1}
I20260807 05:05:27.351015 11617 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44023
I20260807 05:05:27.351239 11617 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:27.351686 11617 heartbeater.cc:511] Master 127.1.17.254:44023 requested a full tablet report, sending...
I20260807 05:05:27.352775 11282 ts_manager.cc:194] Registered new tserver with Master: 591c63cfda704584b3f81c0612fb5552 (127.1.17.194:35957)
W20260807 05:05:27.460438 11620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:27.460676 11620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:27.460714 11620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:27.464685 11620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:27.464736 11620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:27.464847 11620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:05:27.469579 11620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:44023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:27.470664 11620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:27.472082 11620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:27.481926 11630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:27.481966 11627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:27.481952 11628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:27.482308 11620 server_base.cc:1034] running on GCE node
I20260807 05:05:27.482741 11620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:27.483440 11620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:27.484607 11620 hybrid_clock.cc:648] HybridClock initialized: now 1786079127484581 us; error 49 us; skew 500 ppm
I20260807 05:05:27.486531 11620 webserver.cc:460] Webserver started at http://127.1.17.195:34639/ using document root <none> and password file <none>
I20260807 05:05:27.487125 11620 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:27.487188 11620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:27.487424 11620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:27.489547 11620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "18a8be5017e04cc0a45126b80e86a589"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "0146106e58499e6368dd873967ded785"
server_key_iv: "9c16497acb0bfb7ce99742f427f3ee7e"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.490118 11620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "18a8be5017e04cc0a45126b80e86a589"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "0146106e58499e6368dd873967ded785"
server_key_iv: "9c16497acb0bfb7ce99742f427f3ee7e"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.493777 11620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:05:27.496316 11637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.497718 11620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:05:27.497855 11620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "18a8be5017e04cc0a45126b80e86a589"
format_stamp: "Formatted at 2026-08-07 05:05:27 on dist-test-slave-1x4g"
server_key: "0146106e58499e6368dd873967ded785"
server_key_iv: "9c16497acb0bfb7ce99742f427f3ee7e"
server_key_version: "encryptionkey@0"
I20260807 05:05:27.497977 11620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:27.502530 11620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:27.503194 11620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:27.503369 11620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:27.503907 11620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:27.504850 11620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:27.504914 11620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.504984 11620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:27.505035 11620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:27.513785 11620 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:43109
I20260807 05:05:27.513816 11755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:43109 every 8 connection(s)
I20260807 05:05:27.514672 11620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:05:27.520728 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:27.516489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:46253 (local address 127.1.17.254:44023)
0807 05:05:27.516636 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:27.516640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:27.517742 (+ 1102us) server_negotiation.cc:407] Connection header received
0807 05:05:27.519066 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:27.519069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:27.519104 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:27.519179 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:27.520338 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:27.520341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:27.520343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:05:27.520355 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:27.520376 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:05:27.520385 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:27.520478 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:27.520580 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 05:05:27.520618 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:05:27.521592 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:27.516959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44023 (local address 127.1.17.195:46253)
0807 05:05:27.517601 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:27.517641 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:27.518810 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:27.519336 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:27.519343 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:27.519426 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:27.520088 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:05:27.520089 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:27.520129 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:05:27.520194 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:27.520465 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:27.520469 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:27.520473 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:05:27.520588 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 05:05:27.520764 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":116,"threads_started":1}
I20260807 05:05:27.522672 11756 heartbeater.cc:347] Connected to a master server at 127.1.17.254:44023
I20260807 05:05:27.522898 11756 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:27.523294 11756 heartbeater.cc:511] Master 127.1.17.254:44023 requested a full tablet report, sending...
I20260807 05:05:27.523483 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 11620
I20260807 05:05:27.523597 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:05:27.523865 1095 external_mini_cluster.cc:1428] Setting key 2b6c3a447263b44942f7ad134df4fdaf
I20260807 05:05:27.524444 11282 ts_manager.cc:194] Registered new tserver with Master: 18a8be5017e04cc0a45126b80e86a589 (127.1.17.195:43109)
I20260807 05:05:27.525940 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 05:05:27.528683 11322 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:05:27.527794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.1.17.254:44023)
0807 05:05:27.527939 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:27.527943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:27.528104 (+ 161us) server_negotiation.cc:407] Connection header received
0807 05:05:27.528193 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:27.528195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:27.528223 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:27.528283 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:27.528544 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60286: BlockingRecv error: recv got EOF from 127.0.0.1:60286 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:05:27.528874 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 11344
I20260807 05:05:27.539311 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 11481
I20260807 05:05:27.545416 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 11620
I20260807 05:05:27.552350 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 11251
2026-08-07T05:05:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (817 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:05:27 dist-test-slave-1x4g krb5kdc[11766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:05:27 dist-test-slave-1x4g krb5kdc[11766](info): set up 2 sockets
Aug 07 05:05:27 dist-test-slave-1x4g krb5kdc[11766](info): commencing operation
krb5kdc: starting...
W20260807 05:05:28.598661 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:05:28 dist-test-slave-1x4g krb5kdc[11766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:05:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:05:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:05:28.721971 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:40531
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:40531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:28.834318 11780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:28.834573 11780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:28.834613 11780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:28.838562 11780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:05:28.838632 11780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:28.838666 11780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:28.838701 11780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:05:28.838716 11780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:05:28.843730 11780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:40531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:40531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:28.844703 11780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:28.845687 11780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:28.854136 11788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:28.854139 11785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:28.854130 11786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:28.854524 11780 server_base.cc:1034] running on GCE node
I20260807 05:05:28.855118 11780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:28.856222 11780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:28.857411 11780 hybrid_clock.cc:648] HybridClock initialized: now 1786079128857388 us; error 34 us; skew 500 ppm
Aug 07 05:05:28 dist-test-slave-1x4g krb5kdc[11766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079128, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:28.865031 11780 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:05:28.865911 11780 webserver.cc:460] Webserver started at http://127.1.17.254:36923/ using document root <none> and password file <none>
I20260807 05:05:28.866482 11780 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:28.866575 11780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:28.866811 11780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:28.869063 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "9e34f4950c7b47cd9ddb520a474750da"
format_stamp: "Formatted at 2026-08-07 05:05:28 on dist-test-slave-1x4g"
server_key: "52aa35d1e04792bdfde8f28f4b69958f"
server_key_iv: "f47d8ea9999ec72ddb1f170a53580d9a"
server_key_version: "encryptionkey@0"
I20260807 05:05:28.869627 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "9e34f4950c7b47cd9ddb520a474750da"
format_stamp: "Formatted at 2026-08-07 05:05:28 on dist-test-slave-1x4g"
server_key: "52aa35d1e04792bdfde8f28f4b69958f"
server_key_iv: "f47d8ea9999ec72ddb1f170a53580d9a"
server_key_version: "encryptionkey@0"
I20260807 05:05:28.873437 11780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:05:28.875860 11795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:28.877132 11780 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:28.877281 11780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "9e34f4950c7b47cd9ddb520a474750da"
format_stamp: "Formatted at 2026-08-07 05:05:28 on dist-test-slave-1x4g"
server_key: "52aa35d1e04792bdfde8f28f4b69958f"
server_key_iv: "f47d8ea9999ec72ddb1f170a53580d9a"
server_key_version: "encryptionkey@0"
I20260807 05:05:28.877398 11780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:28.895020 11780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:28.898034 11780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:28.898252 11780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:28.905405 11780 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:40531
I20260807 05:05:28.905711 11850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:40531 every 8 connection(s)
I20260807 05:05:28.906466 11780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:05:28.908159 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 11780
I20260807 05:05:28.908284 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:05:28.908619 1095 external_mini_cluster.cc:1428] Setting key 78801ffbca6db897d7c2d8a56143bfa5
I20260807 05:05:28.909744 11851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:28.916312 11851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da: Bootstrap starting.
Aug 07 05:05:28 dist-test-slave-1x4g krb5kdc[11766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:05:28.918954 11851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:28.919781 11851 log.cc:824] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:28.921751 11851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da: No bootstrap required, opened a new log
I20260807 05:05:28.923439 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:28.910159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.1.17.254:40531)
0807 05:05:28.910593 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:28.910620 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:28.910658 (+ 38us) server_negotiation.cc:407] Connection header received
0807 05:05:28.911301 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:28.911317 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:28.911657 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:28.911940 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:28.912907 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:28.913940 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:28.915163 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:28.915659 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:28.915781 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:28.918593 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:28.918598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:28.918611 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:28.918638 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:28.920562 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:28.921084 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:28.921090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:28.921096 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:28.921216 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:28.921519 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:28.921522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:28.921523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:28.921984 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:28.922202 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:28.922483 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 05:05:28.922762 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":120,"threads_started":1}
I20260807 05:05:28.925608 11851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e34f4950c7b47cd9ddb520a474750da" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40531 } }
I20260807 05:05:28.925911 11851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:28.925994 11851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e34f4950c7b47cd9ddb520a474750da, State: Initialized, Role: FOLLOWER
I20260807 05:05:28.926403 11851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e34f4950c7b47cd9ddb520a474750da" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40531 } }
I20260807 05:05:28.926555 11851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:05:28.926641 11851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:05:28.926740 11851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:28.927630 11851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e34f4950c7b47cd9ddb520a474750da" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40531 } }
I20260807 05:05:28.928001 11851 leader_election.cc:302] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e34f4950c7b47cd9ddb520a474750da; no voters:
I20260807 05:05:28.928366 11851 leader_election.cc:288] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:05:28.928488 11856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:28.928750 11856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [term 1 LEADER]: Becoming Leader. State: Replica: 9e34f4950c7b47cd9ddb520a474750da, State: Running, Role: LEADER
I20260807 05:05:28.929136 11856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e34f4950c7b47cd9ddb520a474750da" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40531 } }
I20260807 05:05:28.929653 11851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:05:28.930828 11857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e34f4950c7b47cd9ddb520a474750da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e34f4950c7b47cd9ddb520a474750da" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40531 } } }
I20260807 05:05:28.930968 11857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:28.931315 11858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e34f4950c7b47cd9ddb520a474750da. Latest consensus state: current_term: 1 leader_uuid: "9e34f4950c7b47cd9ddb520a474750da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e34f4950c7b47cd9ddb520a474750da" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40531 } } }
I20260807 05:05:28.931375 11865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:05:28.931416 11858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da [sys.catalog]: This master's current role is: LEADER
I20260807 05:05:28.934053 11865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:05:28.939289 11865 catalog_manager.cc:1347] Generated new cluster ID: d585e2bd8cdf4f33b4643e312fbe5d61
I20260807 05:05:28.939354 11865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:05:28.949595 11865 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:05:28.950748 11865 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:05:28.959280 11865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da: Generated new TSK 0
I20260807 05:05:28.960052 11865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:05:28.992542 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40531
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:29.107215 11877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:29.107518 11877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:29.107594 11877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:29.111481 11877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:29.111559 11877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:29.111644 11877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:05:29.116514 11877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:29.117549 11877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:29.118531 11877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:29.127794 11883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:29.127864 11877 server_base.cc:1034] running on GCE node
W20260807 05:05:29.127813 11882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:29.127813 11887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:29.128427 11877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:29.129071 11877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:29.130257 11877 hybrid_clock.cc:648] HybridClock initialized: now 1786079129130241 us; error 27 us; skew 500 ppm
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[11766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:29.132980 11877 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:05:29.133745 11877 webserver.cc:460] Webserver started at http://127.1.17.193:35609/ using document root <none> and password file <none>
I20260807 05:05:29.134234 11877 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:29.134287 11877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:29.134459 11877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:29.136366 11877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "3cf15d5e672b4fc2a5a10904ce576cc1"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "e08db498f2e81b8a68d23fbd1eae46ec"
server_key_iv: "88b390750529f73795e5fa0d7db08a6f"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.136982 11877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "3cf15d5e672b4fc2a5a10904ce576cc1"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "e08db498f2e81b8a68d23fbd1eae46ec"
server_key_iv: "88b390750529f73795e5fa0d7db08a6f"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.140444 11877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:05:29.142683 11894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.143811 11877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:05:29.143926 11877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "3cf15d5e672b4fc2a5a10904ce576cc1"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "e08db498f2e81b8a68d23fbd1eae46ec"
server_key_iv: "88b390750529f73795e5fa0d7db08a6f"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.144022 11877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:29.160655 11877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:29.163389 11877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:29.163575 11877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:29.164134 11877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:29.164958 11877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:29.165022 11877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.165102 11877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:29.165131 11877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.174024 11877 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:42365
I20260807 05:05:29.174032 12014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:42365 every 8 connection(s)
I20260807 05:05:29.174913 11877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:05:29.179080 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 11877
I20260807 05:05:29.179193 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:05:29.179440 1095 external_mini_cluster.cc:1428] Setting key caa79eb2d8c231a042f8159734846cc6
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[11766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:05:29.190037 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.176616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:45003 (local address 127.1.17.254:40531)
0807 05:05:29.176810 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.176817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.177630 (+ 813us) server_negotiation.cc:407] Connection header received
0807 05:05:29.178922 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.178925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.178994 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.179096 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:29.180527 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.181218 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.183104 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.183539 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.183660 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.186351 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:29.186354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:29.186356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:29.186380 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:29.187928 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.188454 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.188460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.188462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.188542 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.188911 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.188915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.188916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.189107 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:29.189215 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.189735 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.189864 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 05:05:29.190841 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.177036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:40531 (local address 127.1.17.193:45003)
0807 05:05:29.177480 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.177522 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.178708 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.179289 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.179297 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.179735 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:29.180324 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.180340 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.181355 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.181362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.182926 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.182953 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.183690 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.183693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.183840 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.184436 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:29.184454 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:29.186178 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:29.188055 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:29.188060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:29.188076 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:29.188342 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:29.188637 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:29.188640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:29.188642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:29.188828 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:29.189229 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:29.189237 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:29.189488 (+ 251us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.189679 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.190009 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":107,"threads_started":1}
I20260807 05:05:29.192216 12015 heartbeater.cc:347] Connected to a master server at 127.1.17.254:40531
I20260807 05:05:29.192509 12015 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:05:29.193159 12015 heartbeater.cc:511] Master 127.1.17.254:40531 requested a full tablet report, sending...
I20260807 05:05:29.194880 11813 ts_manager.cc:194] Registered new tserver with Master: 3cf15d5e672b4fc2a5a10904ce576cc1 (127.1.17.193:42365)
I20260807 05:05:29.196501 11813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:45003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:05:29.209759 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40531
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:29.317073 12020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:29.317301 12020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:29.317342 12020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:29.321242 12020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:29.321287 12020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:29.321383 12020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:05:29.326360 12020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:29.327252 12020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:29.328208 12020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:29.337908 12025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:29.338006 12027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:29.338081 12020 server_base.cc:1034] running on GCE node
W20260807 05:05:29.337908 12031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:29.338521 12020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:29.339051 12020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:29.340211 12020 hybrid_clock.cc:648] HybridClock initialized: now 1786079129340191 us; error 31 us; skew 500 ppm
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[11766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:29.342939 12020 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:05:29.343806 12020 webserver.cc:460] Webserver started at http://127.1.17.194:36083/ using document root <none> and password file <none>
I20260807 05:05:29.344342 12020 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:29.344399 12020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:29.344646 12020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:29.346637 12020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "31427ac0d49a4dc4a9794aaa273e944f"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "572b8523e5b24b2a2e782217ba4712e9"
server_key_iv: "49eae26d100f20037f163426a752f60d"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.347169 12020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "31427ac0d49a4dc4a9794aaa273e944f"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "572b8523e5b24b2a2e782217ba4712e9"
server_key_iv: "49eae26d100f20037f163426a752f60d"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.350837 12020 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:05:29.353196 12038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.354354 12020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:29.354506 12020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "31427ac0d49a4dc4a9794aaa273e944f"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "572b8523e5b24b2a2e782217ba4712e9"
server_key_iv: "49eae26d100f20037f163426a752f60d"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.354616 12020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:29.365170 12020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:29.367897 12020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:29.368096 12020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:29.368655 12020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:29.369604 12020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:29.369674 12020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.369740 12020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:29.369791 12020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.378919 12020 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:44625
I20260807 05:05:29.378957 12154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:44625 every 8 connection(s)
I20260807 05:05:29.379791 12020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:05:29.385995 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 12020
I20260807 05:05:29.386137 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:05:29.386415 1095 external_mini_cluster.cc:1428] Setting key 7d01af09cf9861000452083d906d38c3
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[11766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:05:29.394363 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.381586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:39869 (local address 127.1.17.254:40531)
0807 05:05:29.381716 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.381721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.382630 (+ 909us) server_negotiation.cc:407] Connection header received
0807 05:05:29.383982 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.383985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.384040 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.384121 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:29.385325 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.385865 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.387253 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.387499 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.387611 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.390337 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:29.390342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:29.390347 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:29.390380 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:29.392076 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.392575 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.392581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.392586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.392680 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.393082 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.393089 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.393090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.393353 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:29.393485 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.393974 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.394124 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:05:29.394917 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.382022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:40531 (local address 127.1.17.194:39869)
0807 05:05:29.382488 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.382531 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.383765 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.384292 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.384299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.384645 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:29.385184 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.385198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.386006 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.386009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.387131 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.387142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.387653 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.387657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.387818 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.388407 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:29.388424 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:29.390174 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:29.392195 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:29.392200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:29.392211 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:29.392474 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:29.392824 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:29.392832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:29.392834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:29.392968 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:29.393494 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:29.393499 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:29.393717 (+ 218us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.393860 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.394101 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
I20260807 05:05:29.396009 12155 heartbeater.cc:347] Connected to a master server at 127.1.17.254:40531
I20260807 05:05:29.396245 12155 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:29.396709 12155 heartbeater.cc:511] Master 127.1.17.254:40531 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:05:29.397766 11813 ts_manager.cc:194] Registered new tserver with Master: 31427ac0d49a4dc4a9794aaa273e944f (127.1.17.194:44625)
I20260807 05:05:29.398326 11813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:39869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:05:29.415565 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40531
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:05:29.526175 12160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:05:29.526430 12160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:05:29.526510 12160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:05:29.530475 12160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:05:29.530540 12160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:05:29.530619 12160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:05:29.535737 12160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:05:29.536746 12160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:05:29.537744 12160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:05:29.547612 12168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:29.547588 12167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:05:29.547588 12170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:05:29.548087 12160 server_base.cc:1034] running on GCE node
I20260807 05:05:29.548523 12160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:05:29.549165 12160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:05:29.550374 12160 hybrid_clock.cc:648] HybridClock initialized: now 1786079129550334 us; error 47 us; skew 500 ppm
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[11766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:05:29.553820 12160 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:05:29.554761 12160 webserver.cc:460] Webserver started at http://127.1.17.195:45969/ using document root <none> and password file <none>
I20260807 05:05:29.555365 12160 fs_manager.cc:359] Metadata directory not provided
I20260807 05:05:29.555464 12160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:05:29.555708 12160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:05:29.557964 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "8593004d90e64def8a217cf97761c497"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "fe954d0e2cec5b4b6d1f6426452133c7"
server_key_iv: "8237ead5bf4759f0a10fa8d7498b17e6"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.558604 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "8593004d90e64def8a217cf97761c497"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "fe954d0e2cec5b4b6d1f6426452133c7"
server_key_iv: "8237ead5bf4759f0a10fa8d7498b17e6"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.562635 12160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 05:05:29.565232 12177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.566419 12160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:05:29.566543 12160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "8593004d90e64def8a217cf97761c497"
format_stamp: "Formatted at 2026-08-07 05:05:29 on dist-test-slave-1x4g"
server_key: "fe954d0e2cec5b4b6d1f6426452133c7"
server_key_iv: "8237ead5bf4759f0a10fa8d7498b17e6"
server_key_version: "encryptionkey@0"
I20260807 05:05:29.566648 12160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:05:29.582568 12160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:05:29.585738 12160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:05:29.585943 12160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:05:29.586598 12160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:05:29.587402 12160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:05:29.587453 12160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.587517 12160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:05:29.587565 12160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:05:29.596431 12160 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:34227
I20260807 05:05:29.596464 12294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:34227 every 8 connection(s)
I20260807 05:05:29.597396 12160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:05:29.601644 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 12160
I20260807 05:05:29.601737 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:05:29.602030 1095 external_mini_cluster.cc:1428] Setting key d4bf672406c6716147354e0c6f0b19ed
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[11766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079129, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:05:29.611174 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.599144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:42447 (local address 127.1.17.254:40531)
0807 05:05:29.599292 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.599295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.600130 (+ 835us) server_negotiation.cc:407] Connection header received
0807 05:05:29.601596 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.601599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.601646 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.601739 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:29.603175 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.603747 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.605129 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.605388 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.605512 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.607853 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:29.607863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:29.607867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:29.607890 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:29.609148 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.609681 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.609686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.609688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.609769 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.610121 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.610126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.610128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.610304 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:29.610415 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.610883 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.611005 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:05:29.611905 12298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.599510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:40531 (local address 127.1.17.195:42447)
0807 05:05:29.599971 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.600013 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.601305 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.601966 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.601973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.602416 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:05:29.603036 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.603053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.603901 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.603905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.605001 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.605012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.605548 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.605553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.605735 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.606267 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:05:29.606284 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:05:29.607694 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:05:29.609265 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:29.609271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:29.609284 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:29.609574 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:29.609871 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:05:29.609874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:05:29.609876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:05:29.610023 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:05:29.610414 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:05:29.610420 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:05:29.610654 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.610839 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.611147 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":134,"threads_started":1}
I20260807 05:05:29.613011 12295 heartbeater.cc:347] Connected to a master server at 127.1.17.254:40531
I20260807 05:05:29.613265 12295 heartbeater.cc:464] Registering TS with master...
I20260807 05:05:29.613742 12295 heartbeater.cc:511] Master 127.1.17.254:40531 requested a full tablet report, sending...
I20260807 05:05:29.614717 11813 ts_manager.cc:194] Registered new tserver with Master: 8593004d90e64def8a217cf97761c497 (127.1.17.195:34227)
I20260807 05:05:29.615350 11813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:42447
I20260807 05:05:29.616624 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:05:29.625255 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.618619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32814 (local address 127.1.17.254:40531)
0807 05:05:29.618724 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.618727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.618909 (+ 182us) server_negotiation.cc:407] Connection header received
0807 05:05:29.619001 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.619003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.619037 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.619102 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:05:29.619728 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.620199 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.621077 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.621354 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.621444 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.622652 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:05:29.622655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:05:29.622657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:05:29.622677 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:05:29.623841 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.624208 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.624210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.624211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.624282 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:05:29.624512 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:05:29.624514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:05:29.624515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:05:29.624700 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:05:29.624858 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.625066 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.625153 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 05:05:29.627755 11813 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:05:29.630863 11813 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:05:29.645588 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.639134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41482 (local address 127.1.17.193:42365)
0807 05:05:29.639742 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.639764 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.640222 (+ 458us) server_negotiation.cc:407] Connection header received
0807 05:05:29.640448 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.640455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.640683 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.640834 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:29.641301 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.642443 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.643923 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.644714 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.644870 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.645050 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.645307 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.645407 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":111,"threads_started":1}
I20260807 05:05:29.645627 12310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.639481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:34227 (local address 127.0.0.1:47044)
0807 05:05:29.640073 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.640127 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.640256 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.640985 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.640988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.641020 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:29.641126 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.641131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.642770 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.642776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.643951 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.643959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.644973 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.644977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.645124 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.645141 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.645254 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.645407 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":74,"spinlock_wait_cycles":896,"thread_start_us":104,"threads_started":1}
I20260807 05:05:29.646030 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.639665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:44625 (local address 127.0.0.1:42836)
0807 05:05:29.640685 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.640700 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.640822 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.641300 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.641302 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.641318 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:29.641417 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.641423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.642340 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.642344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.644423 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.644428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.645651 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.645655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.645749 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.645770 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.645843 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.645917 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":25,"thread_start_us":118,"threads_started":1}
I20260807 05:05:29.646029 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.638981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:42365 (local address 127.0.0.1:41482)
0807 05:05:29.640073 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.640127 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.640283 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.640974 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.640980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.641012 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:29.641162 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.641168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.642565 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.642568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.643795 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.643806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.644863 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.644866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.645058 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.645114 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.645245 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.645425 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":111,"spinlock_wait_cycles":2304,"thread_start_us":65,"threads_started":1}
I20260807 05:05:29.646600 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.640164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.1.17.194:44625)
0807 05:05:29.640686 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.640701 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.640762 (+ 61us) server_negotiation.cc:407] Connection header received
0807 05:05:29.640969 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.640974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.641177 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.641325 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:29.641517 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.642216 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.644530 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.645323 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.645419 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.645585 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.646416 (+ 831us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.646493 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":121,"threads_started":1}
I20260807 05:05:29.646705 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.639669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47044 (local address 127.1.17.195:34227)
0807 05:05:29.640523 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.640536 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.640557 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:05:29.640622 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.640627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.640829 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.641282 (+ 453us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:29.641442 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.642604 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.644062 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.644843 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.645502 (+ 659us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.645670 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.645732 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.645818 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":164,"threads_started":1}
I20260807 05:05:29.649171 11946 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2b8385939348e0b4e2c30bf79ce6a9 (DEFAULT_TABLE table=test-table [id=d38e3896f92343519c0395146056aac2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:29.649351 12087 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2b8385939348e0b4e2c30bf79ce6a9 (DEFAULT_TABLE table=test-table [id=d38e3896f92343519c0395146056aac2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:29.649358 12227 tablet_service.cc:1467] Processing CreateTablet for tablet 6b2b8385939348e0b4e2c30bf79ce6a9 (DEFAULT_TABLE table=test-table [id=d38e3896f92343519c0395146056aac2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:05:29.650193 11946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2b8385939348e0b4e2c30bf79ce6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:29.650377 12227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2b8385939348e0b4e2c30bf79ce6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:29.650370 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b2b8385939348e0b4e2c30bf79ce6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:05:29.656057 12313 tablet_bootstrap.cc:492] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1: Bootstrap starting.
I20260807 05:05:29.657698 12316 tablet_bootstrap.cc:492] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Bootstrap starting.
I20260807 05:05:29.658974 12313 tablet_bootstrap.cc:654] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:29.659958 12316 tablet_bootstrap.cc:654] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:29.660106 12313 log.cc:824] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:29.660758 12314 tablet_bootstrap.cc:492] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Bootstrap starting.
I20260807 05:05:29.660781 12316 log.cc:824] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:29.662436 12313 tablet_bootstrap.cc:492] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1: No bootstrap required, opened a new log
I20260807 05:05:29.662653 12313 ts_tablet_manager.cc:1397] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 05:05:29.662905 12316 tablet_bootstrap.cc:492] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: No bootstrap required, opened a new log
I20260807 05:05:29.663105 12316 ts_tablet_manager.cc:1397] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:05:29.663125 12314 tablet_bootstrap.cc:654] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Neither blocks nor log segments found. Creating new log.
I20260807 05:05:29.663772 12314 log.cc:824] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Log is configured to *not* fsync() on all Append() calls
I20260807 05:05:29.665650 12314 tablet_bootstrap.cc:492] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: No bootstrap required, opened a new log
I20260807 05:05:29.665839 12314 ts_tablet_manager.cc:1397] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 05:05:29.667097 12316 raft_consensus.cc:348] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.667390 12316 raft_consensus.cc:374] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:29.667245 12313 raft_consensus.cc:348] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.667477 12316 raft_consensus.cc:729] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31427ac0d49a4dc4a9794aaa273e944f, State: Initialized, Role: FOLLOWER
I20260807 05:05:29.667536 12313 raft_consensus.cc:374] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:29.667625 12313 raft_consensus.cc:729] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf15d5e672b4fc2a5a10904ce576cc1, State: Initialized, Role: FOLLOWER
I20260807 05:05:29.668023 12316 consensus_queue.cc:260] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.668023 12313 consensus_queue.cc:260] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.668968 12155 heartbeater.cc:503] Master 127.1.17.254:40531 was elected leader, sending a full tablet report...
I20260807 05:05:29.669082 12316 ts_tablet_manager.cc:1428] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:05:29.669082 12313 ts_tablet_manager.cc:1428] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:05:29.669142 12314 raft_consensus.cc:348] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.669384 12314 raft_consensus.cc:374] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:05:29.669430 12314 raft_consensus.cc:729] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8593004d90e64def8a217cf97761c497, State: Initialized, Role: FOLLOWER
I20260807 05:05:29.669811 12314 consensus_queue.cc:260] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.670213 12015 heartbeater.cc:503] Master 127.1.17.254:40531 was elected leader, sending a full tablet report...
I20260807 05:05:29.670805 12314 ts_tablet_manager.cc:1428] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:05:29.671128 12295 heartbeater.cc:503] Master 127.1.17.254:40531 was elected leader, sending a full tablet report...
W20260807 05:05:29.675940 12016 tablet.cc:2369] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:05:29.828540 12320 raft_consensus.cc:482] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:05:29.828806 12320 raft_consensus.cc:504] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.830158 12320 leader_election.cc:288] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31427ac0d49a4dc4a9794aaa273e944f (127.1.17.194:44625), 8593004d90e64def8a217cf97761c497 (127.1.17.195:34227)
I20260807 05:05:29.834237 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.830748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:39123 (local address 127.1.17.195:34227)
0807 05:05:29.830892 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.830896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.831024 (+ 128us) server_negotiation.cc:407] Connection header received
0807 05:05:29.831187 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.831190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.831237 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.831320 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:29.831597 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.832137 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.833246 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.833648 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.833781 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.833836 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.834024 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.834098 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 05:05:29.834257 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.830453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:34227 (local address 127.1.17.193:39123)
0807 05:05:29.830916 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.830930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.831060 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.831329 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.831332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.831348 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:29.831476 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.831481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.832252 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.832256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.833116 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.833125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.833775 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.833779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.833884 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.833915 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.834005 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.834076 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":3,"thread_start_us":149,"threads_started":1}
I20260807 05:05:29.834299 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.830342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:44625 (local address 127.1.17.193:34673)
0807 05:05:29.830693 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.830709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:05:29.830865 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:05:29.831191 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:05:29.831194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:05:29.831212 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:05:29.831333 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.831339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.832172 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.832177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.833136 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:05:29.833146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.833799 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.833802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:05:29.833949 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.833968 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:05:29.834058 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 05:05:29.834135 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":35,"thread_start_us":128,"threads_started":1}
I20260807 05:05:29.834282 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.830508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:34673 (local address 127.1.17.194:44625)
0807 05:05:29.830664 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.830669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.830816 (+ 147us) server_negotiation.cc:407] Connection header received
0807 05:05:29.831033 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.831036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.831081 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.831167 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:05:29.831460 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.832062 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.833263 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.833667 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.833797 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.833852 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.834079 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.834158 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 05:05:29.834941 12107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2b8385939348e0b4e2c30bf79ce6a9" candidate_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31427ac0d49a4dc4a9794aaa273e944f" is_pre_election: true
I20260807 05:05:29.835173 12247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2b8385939348e0b4e2c30bf79ce6a9" candidate_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593004d90e64def8a217cf97761c497" is_pre_election: true
I20260807 05:05:29.835289 12107 raft_consensus.cc:2454] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cf15d5e672b4fc2a5a10904ce576cc1 in term 0.
I20260807 05:05:29.835454 12247 raft_consensus.cc:2454] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cf15d5e672b4fc2a5a10904ce576cc1 in term 0.
I20260807 05:05:29.835819 11897 leader_election.cc:302] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31427ac0d49a4dc4a9794aaa273e944f, 3cf15d5e672b4fc2a5a10904ce576cc1; no voters:
I20260807 05:05:29.836148 12320 raft_consensus.cc:2781] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:05:29.836228 12320 raft_consensus.cc:482] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:05:29.836258 12320 raft_consensus.cc:3037] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:29.837369 12320 raft_consensus.cc:504] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.837724 12320 leader_election.cc:288] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [CANDIDATE]: Term 1 election: Requested vote from peers 31427ac0d49a4dc4a9794aaa273e944f (127.1.17.194:44625), 8593004d90e64def8a217cf97761c497 (127.1.17.195:34227)
I20260807 05:05:29.838078 12107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2b8385939348e0b4e2c30bf79ce6a9" candidate_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31427ac0d49a4dc4a9794aaa273e944f"
I20260807 05:05:29.838125 12247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b2b8385939348e0b4e2c30bf79ce6a9" candidate_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8593004d90e64def8a217cf97761c497"
I20260807 05:05:29.838207 12107 raft_consensus.cc:3037] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:29.838270 12247 raft_consensus.cc:3037] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:05:29.839257 12107 raft_consensus.cc:2454] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cf15d5e672b4fc2a5a10904ce576cc1 in term 1.
I20260807 05:05:29.839401 12247 raft_consensus.cc:2454] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cf15d5e672b4fc2a5a10904ce576cc1 in term 1.
I20260807 05:05:29.839550 11897 leader_election.cc:302] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31427ac0d49a4dc4a9794aaa273e944f, 3cf15d5e672b4fc2a5a10904ce576cc1; no voters:
I20260807 05:05:29.839733 12320 raft_consensus.cc:2781] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:05:29.839986 12320 raft_consensus.cc:686] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [term 1 LEADER]: Becoming Leader. State: Replica: 3cf15d5e672b4fc2a5a10904ce576cc1, State: Running, Role: LEADER
I20260807 05:05:29.840312 12320 consensus_queue.cc:237] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } }
I20260807 05:05:29.843423 11810 catalog_manager.cc:5495] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cf15d5e672b4fc2a5a10904ce576cc1 (127.1.17.193). New cstate: current_term: 1 leader_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf15d5e672b4fc2a5a10904ce576cc1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 } health_report { overall_health: UNKNOWN } } }
W20260807 05:05:29.848312 12296 tablet.cc:2369] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:05:29.881034 12156 tablet.cc:2369] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:05:29.881676 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.877596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.1.17.193:42365)
0807 05:05:29.877734 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:05:29.877737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:05:29.877752 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:05:29.877904 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:05:29.877906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:05:29.877980 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:05:29.878072 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:05:29.878338 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.878975 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.879840 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:05:29.880090 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:05:29.880162 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:05:29.880535 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:05:29.881150 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:05:29.881272 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 05:05:29.881416 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 05:05:29.881484 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:05:29.887562 12247 raft_consensus.cc:1261] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 1 FOLLOWER]: Refusing update from remote peer 3cf15d5e672b4fc2a5a10904ce576cc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:29.887564 12107 raft_consensus.cc:1261] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 1 FOLLOWER]: Refusing update from remote peer 3cf15d5e672b4fc2a5a10904ce576cc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:05:29.888322 12320 consensus_queue.cc:1035] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31427ac0d49a4dc4a9794aaa273e944f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:29.888562 12325 consensus_queue.cc:1035] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 3cf15d5e672b4fc2a5a10904ce576cc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8593004d90e64def8a217cf97761c497" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 34227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:05:29.896167 12330 mvcc.cc:205] Tried to move back new op lower bound from 7315780116011810816 to 7315780115828797440. Current Snapshot: MvccSnapshot[applied={T|T < 7315780116011810816}]
I20260807 05:05:29.897790 12329 mvcc.cc:205] Tried to move back new op lower bound from 7315780116011810816 to 7315780115828797440. Current Snapshot: MvccSnapshot[applied={T|T < 7315780116011810816}]
I20260807 05:05:29.903924 11810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:29.904096 11810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:05:29.906646 11810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e34f4950c7b47cd9ddb520a474750da: Sending DeleteTablet for 3 replicas of tablet 6b2b8385939348e0b4e2c30bf79ce6a9
I20260807 05:05:29.907610 12227 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2b8385939348e0b4e2c30bf79ce6a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:29 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:47044
I20260807 05:05:29.908123 12340 tablet_replica.cc:330] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: stopping tablet replica
I20260807 05:05:29.908324 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 11877
I20260807 05:05:29.908532 12340 raft_consensus.cc:2229] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:29.909272 12340 raft_consensus.cc:2258] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:29.909725 12087 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b2b8385939348e0b4e2c30bf79ce6a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:05:29 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:42836
I20260807 05:05:29.910146 12341 tablet_replica.cc:330] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: stopping tablet replica
I20260807 05:05:29.910499 12341 raft_consensus.cc:2229] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:05:29.910797 12341 raft_consensus.cc:2258] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:05:29.912161 12341 ts_tablet_manager.cc:1905] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:29.912971 12340 ts_tablet_manager.cc:1905] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:05:29.914919 12341 ts_tablet_manager.cc:1918] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:29.915050 12341 log.cc:1196] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/6b2b8385939348e0b4e2c30bf79ce6a9
I20260807 05:05:29.915410 12341 ts_tablet_manager.cc:1939] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 31427ac0d49a4dc4a9794aaa273e944f: Deleting consensus metadata
I20260807 05:05:29.916551 11798 catalog_manager.cc:4841] TS 31427ac0d49a4dc4a9794aaa273e944f (127.1.17.194:44625): tablet 6b2b8385939348e0b4e2c30bf79ce6a9 (table test-table [id=d38e3896f92343519c0395146056aac2]) successfully deleted
I20260807 05:05:29.916752 12340 ts_tablet_manager.cc:1918] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:05:29.916900 12340 log.cc:1196] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/6b2b8385939348e0b4e2c30bf79ce6a9
I20260807 05:05:29.917294 12340 ts_tablet_manager.cc:1939] T 6b2b8385939348e0b4e2c30bf79ce6a9 P 8593004d90e64def8a217cf97761c497: Deleting consensus metadata
I20260807 05:05:29.918569 11796 catalog_manager.cc:4841] TS 8593004d90e64def8a217cf97761c497 (127.1.17.195:34227): tablet 6b2b8385939348e0b4e2c30bf79ce6a9 (table test-table [id=d38e3896f92343519c0395146056aac2]) successfully deleted
W20260807 05:05:29.919606 11798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:42365 (error 108)
I20260807 05:05:29.920457 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 12020
I20260807 05:05:29.920651 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:05:29.920312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:42365 (local address 127.0.0.1:41486)
0807 05:05:29.920463 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 05:05:29.920551 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 05:05:29.922726 11798 catalog_manager.cc:4584] TS 3cf15d5e672b4fc2a5a10904ce576cc1 (127.1.17.193:42365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b2b8385939348e0b4e2c30bf79ce6a9: Network error: Client connection negotiation failed: client connection to 127.1.17.193:42365: connect: Connection refused (error 111)
I20260807 05:05:29.927569 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 12160
I20260807 05:05:29.934209 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 11780
2026-08-07T05:05:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2383 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[12345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[12345](info): set up 2 sockets
Aug 07 05:05:29 dist-test-slave-1x4g krb5kdc[12345](info): commencing operation
krb5kdc: starting...
W20260807 05:05:31.002869 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:05:31 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:05:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/rangeradmin_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/rangeradmin_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/rangerlookup_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/rangerlookup_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
I20260807 05:05:31.169972 1095 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXAtKix/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/postgres -l logfile start
2026-08-07 05:05:33.624 UTC [12375] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 05:05:33.624 UTC [12375] LOG: listening on IPv4 address "127.1.17.212", port 50537
2026-08-07 05:05:33.627 UTC [12375] LOG: listening on Unix socket "/tmp/.s.PGSQL.50537"
2026-08-07 05:05:33.648 UTC [12378] LOG: database system was shut down at 2026-08-07 05:05:32 UTC
2026-08-07 05:05:33.652 UTC [12375] LOG: database system is ready to accept connections
I20260807 05:05:34.618738 1095 mini_postgres.cc:96] Postgres bound to 50537
2026-08-07 05:05:34.626 UTC [12386] FATAL: database "slave" does not exist
127.1.17.212:50537 - accepting connections
I20260807 05:05:34.626861 1095 mini_ranger.cc:161] Starting Ranger...
I20260807 05:05:34.643661 1095 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 05:05:35.122366 1095 mini_ranger.cc:87] Created ranger Postgres database
I20260807 05:05:35.122474 1095 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-admin
2026-08-07 05:05:35,400 [I] DB FLAVOR :POSTGRES
2026-08-07 05:05:35,401 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:05:35,401 [I] Checking connection..
2026-08-07 05:05:35,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 05:05:35,653 [I] Checking connection passed.
2026-08-07 05:05:35,653 [I] --------- Verifying version history table ---------
2026-08-07 05:05:35,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:05:35,898 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:05:35,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:05:36,133 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:05:36,133 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 05:05:36,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 05:05:36.372 UTC [12454] WARNING: there is no transaction in progress
2026-08-07 05:05:36,379 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 05:05:36,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:05:36,630 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 05:05:36,630 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:05:36,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 05:05:36,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 05:05:37,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-07 05:05:37,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:05:37,621 [I] Table x_portal_user does not exist in database ranger
2026-08-07 05:05:37,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:05:37,847 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 05:05:37,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:05:38,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:05:38,295 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 05:05:38,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 05:05:39.244 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.261 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.276 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.291 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.307 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.325 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.378 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.384 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.398 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.408 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.418 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.431 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.440 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.447 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.453 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:39.459 UTC [12564] WARNING: there is no transaction in progress
2026-08-07 05:05:40,048 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 05:05:40,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 05:05:40,275 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 05:05:40,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:05:40,517 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 05:05:40,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:05:40,754 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 05:05:40,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:05:40,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:05:41,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-07 05:05:41,436 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 05:05:41,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 05:05:41,690 [I] DB_PATCHES have already been applied
I20260807 05:05:41.696069 1095 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXAtKix/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 05:05:41.696149 1095 mini_ranger.cc:193] Using host: 127.1.17.212
I20260807 05:05:41.700664 1095 mini_ranger.cc:227] Ranger admin URL: http://127.1.17.212:47883
Aug 07, 2026 5:05:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:05:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:05:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 5:05:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 5:05:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 5:05:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 5:05:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 05:05:42.854307 1095 mini_ranger.cc:160] Time spent starting Ranger: real 8.227s user 0.000s sys 0.013s
Aug 07, 2026 5:06:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 5:06:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 5:06:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 5:06:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 5:06:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 05:06:03.840190 1095 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/rangerkms_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/rangerkms_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.17.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.17.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 05:06:03 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079163, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 05:06:03.931989 1095 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 05:06:03.941983 1095 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 05:06:04.845932 1095 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 05:06:04.846032 1095 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms
I20260807 05:06:04.846128 1095 mini_ranger_kms.cc:227] Using postgres at 127.1.17.212:50537
2026-08-07 05:06:05,199 [I] DB FLAVOR :POSTGRES
2026-08-07 05:06:05,200 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:06:05,200 [I] Checking connection
2026-08-07 05:06:05,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 05:06:05,453 [I] connection success
2026-08-07 05:06:05,453 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 05:06:05,453 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 05:06:05,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 05:06:05,698 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 05:06:05,698 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:06:05,698 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 05:06:05,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:50537/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 05:06:05,943 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 05:06:06.382210 1095 mini_ranger_kms.cc:310] Created kms service
I20260807 05:06:06.546986 1095 mini_ranger_kms.cc:326] Created kudu user
I20260807 05:06:06.591728 1095 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 05:06:06 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079163, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:06.980618 1095 mini_ranger_kms.cc:384] Added ranger policy
I20260807 05:06:06.980880 1095 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 05:06:06.980937 1095 mini_ranger_kms.cc:240] Using host: 127.1.17.212
I20260807 05:06:06.985647 1095 mini_ranger_kms.cc:276] Ranger KMS PID: 12750
I20260807 05:06:06.985797 1095 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.17.212:40179
Aug 07, 2026 5:06:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:06:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:06:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 5:06:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 5:06:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 5:06:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 05:06:09.237995 1095 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.306s user 0.014s sys 0.323s
I20260807 05:06:09.238129 1095 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 05:06:09.238242 1095 mini_ranger_kms.cc:401] 127.1.17.212:40179/kms/v1/keys
Aug 07 05:06:09 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079169, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.17.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 05:06:09 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079169, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.17.212@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
Aug 07 05:06:10 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079169, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.17.212@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
Aug 07 05:06:12 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:06:12.609853 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38711
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:38711
--ranger_config_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:40179/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:06:12.722082 12801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:06:12.722347 12801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:06:12.722432 12801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:06:12.726280 12801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:06:12.726349 12801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:06:12.726383 12801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:06:12.726436 12801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:06:12.726466 12801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:06:12.731689 12801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.17.212:40179/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:38711
--ranger_config_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:06:12.732702 12801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:06:12.733716 12801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:06:12.742312 12807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:12.742509 12801 server_base.cc:1034] running on GCE node
W20260807 05:06:12.742316 12809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:12.742326 12806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:12.743222 12801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:06:12.744338 12801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:06:12.745518 12801 hybrid_clock.cc:648] HybridClock initialized: now 1786079172745501 us; error 37 us; skew 500 ppm
Aug 07 05:06:12 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079172, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:06:12.748347 12801 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:06:12.749305 12801 webserver.cc:460] Webserver started at http://127.1.17.254:36003/ using document root <none> and password file <none>
I20260807 05:06:12.749842 12801 fs_manager.cc:359] Metadata directory not provided
I20260807 05:06:12.749895 12801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:06:12.750123 12801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:06:12 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079172, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:12.810781 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "bba421f53ee347039a608abcec54e495"
format_stamp: "Formatted at 2026-08-07 05:06:12 on dist-test-slave-1x4g"
server_key: "7025e7a5d4b5dd8e292cd9cde64063f2"
server_key_iv: "4d72a92053e315d9d30977e44a3e69c6"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:12.811407 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "bba421f53ee347039a608abcec54e495"
format_stamp: "Formatted at 2026-08-07 05:06:12 on dist-test-slave-1x4g"
server_key: "7025e7a5d4b5dd8e292cd9cde64063f2"
server_key_iv: "4d72a92053e315d9d30977e44a3e69c6"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:12.815428 12801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:06:12.833829 12817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:12.835830 12801 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 05:06:12.836545 12801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "bba421f53ee347039a608abcec54e495"
format_stamp: "Formatted at 2026-08-07 05:06:12 on dist-test-slave-1x4g"
server_key: "7025e7a5d4b5dd8e292cd9cde64063f2"
server_key_iv: "4d72a92053e315d9d30977e44a3e69c6"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:12.836815 12801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:06:12.862331 12801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:06:12.865420 12801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:06:12.865643 12801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:06:12.873826 12801 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:38711
I20260807 05:06:12.873880 12873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:38711 every 8 connection(s)
I20260807 05:06:12.874925 12801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:06:12.878042 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 12801
I20260807 05:06:12.878181 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:06:12.878465 1095 external_mini_cluster.cc:1428] Setting key 5a0fcd8ffe9ff7a40306f3e7cc6a49d8
I20260807 05:06:12.878513 12874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 05:06:12 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:06:12.886906 12874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495: Bootstrap starting.
I20260807 05:06:12.889415 12874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495: Neither blocks nor log segments found. Creating new log.
I20260807 05:06:12.890470 12874 log.cc:824] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495: Log is configured to *not* fsync() on all Append() calls
I20260807 05:06:12.891817 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:12.879835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.1.17.254:38711)
0807 05:06:12.880260 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:12.880274 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:12.880311 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:06:12.881048 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:12.881066 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:12.881383 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:12.881732 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:06:12.882533 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:12.883496 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:12.884320 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:12.884835 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:12.885007 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:12.887508 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:06:12.887513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:06:12.887524 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:06:12.887548 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:06:12.889136 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:12.889591 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:12.889608 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:12.889612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:12.889710 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:12.890005 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:12.890008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:12.890009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:12.890356 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:06:12.890574 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:12.890877 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 05:06:12.891143 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":136,"threads_started":1}
I20260807 05:06:12.892505 12874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495: No bootstrap required, opened a new log
I20260807 05:06:12.895702 12874 raft_consensus.cc:348] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba421f53ee347039a608abcec54e495" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38711 } }
I20260807 05:06:12.895932 12874 raft_consensus.cc:374] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:06:12.896010 12874 raft_consensus.cc:729] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bba421f53ee347039a608abcec54e495, State: Initialized, Role: FOLLOWER
I20260807 05:06:12.896476 12874 consensus_queue.cc:260] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba421f53ee347039a608abcec54e495" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38711 } }
I20260807 05:06:12.896700 12874 raft_consensus.cc:388] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:06:12.896806 12874 raft_consensus.cc:482] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:06:12.896906 12874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:06:12.897784 12874 raft_consensus.cc:504] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba421f53ee347039a608abcec54e495" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38711 } }
I20260807 05:06:12.898113 12874 leader_election.cc:302] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bba421f53ee347039a608abcec54e495; no voters:
I20260807 05:06:12.898414 12874 leader_election.cc:288] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:06:12.898523 12879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:06:12.898793 12879 raft_consensus.cc:686] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [term 1 LEADER]: Becoming Leader. State: Replica: bba421f53ee347039a608abcec54e495, State: Running, Role: LEADER
I20260807 05:06:12.899196 12879 consensus_queue.cc:237] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba421f53ee347039a608abcec54e495" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38711 } }
I20260807 05:06:12.899701 12874 sys_catalog.cc:564] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:06:12.900872 12881 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bba421f53ee347039a608abcec54e495. Latest consensus state: current_term: 1 leader_uuid: "bba421f53ee347039a608abcec54e495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba421f53ee347039a608abcec54e495" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38711 } } }
I20260807 05:06:12.900993 12880 sys_catalog.cc:455] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bba421f53ee347039a608abcec54e495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bba421f53ee347039a608abcec54e495" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38711 } } }
I20260807 05:06:12.901108 12881 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [sys.catalog]: This master's current role is: LEADER
I20260807 05:06:12.901161 12880 sys_catalog.cc:458] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495 [sys.catalog]: This master's current role is: LEADER
I20260807 05:06:12.901531 12888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:06:12.904239 12888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:06:12.904767 12874 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 05:06:12.912727 12888 catalog_manager.cc:1347] Generated new cluster ID: 50a89e0bd6804e6687a2235375119360
I20260807 05:06:12.912814 12888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:06:12.944025 12888 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:06:12.945243 12888 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:06:12.950356 12888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495: Generated new TSK 0
I20260807 05:06:12.951116 12888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:06:13.649909 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:40179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38711
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:06:13.765498 12920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:06:13.765734 12920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:06:13.765817 12920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:06:13.769889 12920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:06:13.769965 12920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:06:13.770076 12920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:06:13.775348 12920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.17.212:40179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:06:13.776302 12920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:06:13.777568 12920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:06:13.787341 12925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:13.787360 12926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:13.787544 12928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:13.787834 12920 server_base.cc:1034] running on GCE node
I20260807 05:06:13.788268 12920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:06:13.788903 12920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:06:13.790084 12920 hybrid_clock.cc:648] HybridClock initialized: now 1786079173790055 us; error 49 us; skew 500 ppm
Aug 07 05:06:13 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079173, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:06:13.793149 12920 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:06:13.794245 12920 webserver.cc:460] Webserver started at http://127.1.17.193:35085/ using document root <none> and password file <none>
I20260807 05:06:13.795044 12920 fs_manager.cc:359] Metadata directory not provided
I20260807 05:06:13.795147 12920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:06:13.795403 12920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:06:13 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079173, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:13.811190 12920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "1705c2102cdd406eacf5c52446c72058"
format_stamp: "Formatted at 2026-08-07 05:06:13 on dist-test-slave-1x4g"
server_key: "f9df9c54176cfa05e713399036bd86f0"
server_key_iv: "60f0adc7d848535ad6f245dbd52e18cb"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:13.812196 12920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "1705c2102cdd406eacf5c52446c72058"
format_stamp: "Formatted at 2026-08-07 05:06:13 on dist-test-slave-1x4g"
server_key: "f9df9c54176cfa05e713399036bd86f0"
server_key_iv: "60f0adc7d848535ad6f245dbd52e18cb"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:13.817185 12920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 05:06:13.841958 12937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:13.844162 12920 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.003s sys 0.000s
I20260807 05:06:13.844360 12920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "1705c2102cdd406eacf5c52446c72058"
format_stamp: "Formatted at 2026-08-07 05:06:13 on dist-test-slave-1x4g"
server_key: "f9df9c54176cfa05e713399036bd86f0"
server_key_iv: "60f0adc7d848535ad6f245dbd52e18cb"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:13.844548 12920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:06:13.874356 12920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:06:13.878284 12920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:06:13.878557 12920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:06:13.879391 12920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:06:13.880554 12920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:06:13.880699 12920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:13.880877 12920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:06:13.880990 12920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:13.936506 12920 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:46249
I20260807 05:06:13.939260 12920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:06:13.941067 13054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:46249 every 8 connection(s)
I20260807 05:06:13.948732 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 12920
I20260807 05:06:13.948894 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:06:13.949219 1095 external_mini_cluster.cc:1428] Setting key d3f5b67e3d46d02fcd3913ba1c97acda
Aug 07 05:06:13 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079173, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:06:13.964900 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:13.944068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:50161 (local address 127.1.17.254:38711)
0807 05:06:13.944349 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:13.944358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:13.945338 (+ 980us) server_negotiation.cc:407] Connection header received
0807 05:06:13.946465 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:13.946469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:13.946536 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:13.946657 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:06:13.947917 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:13.948596 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:13.950371 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:13.950762 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:13.950922 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:13.959159 (+ 8237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:06:13.959163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:06:13.959166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:06:13.959195 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:06:13.961304 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:13.962012 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:13.962016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:13.962018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:13.962116 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:13.963515 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:13.963519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:13.963521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:13.963767 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:06:13.963851 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:13.964560 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 05:06:13.964688 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
I20260807 05:06:13.967042 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:13.944622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38711 (local address 127.1.17.193:50161)
0807 05:06:13.945175 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:13.945233 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:13.946221 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:13.946850 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:13.946858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:13.947195 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:06:13.947757 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:13.947771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:13.948835 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:13.948843 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:13.950228 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:13.950238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:13.953937 (+ 3699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:13.953941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:13.954143 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:13.956971 (+ 2828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:06:13.956994 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:06:13.958950 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:06:13.961477 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:06:13.961484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:06:13.961500 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:06:13.961882 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:06:13.962230 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:06:13.962233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:06:13.962235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:06:13.962424 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:06:13.963923 (+ 1499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:06:13.963929 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:06:13.964268 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 05:06:13.965413 (+ 1145us) client_negotiation.cc:240] Negotiation successful
0807 05:06:13.965809 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":200,"threads_started":1}
I20260807 05:06:13.968886 13055 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38711
I20260807 05:06:13.969144 13055 heartbeater.cc:464] Registering TS with master...
I20260807 05:06:13.969689 13055 heartbeater.cc:511] Master 127.1.17.254:38711 requested a full tablet report, sending...
I20260807 05:06:13.971416 12835 ts_manager.cc:194] Registered new tserver with Master: 1705c2102cdd406eacf5c52446c72058 (127.1.17.193:46249)
I20260807 05:06:13.973142 12835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:50161
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:06:14.002069 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:40179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38711
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 05:06:14.174402 13063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:06:14.174794 13063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:06:14.174885 13063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:06:14.180078 13063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:06:14.180228 13063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:06:14.180377 13063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:06:14.188360 13063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.17.212:40179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:06:14.189985 13063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:06:14.191428 13063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:06:14.205651 13072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:14.206072 13070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:14.207024 13063 server_base.cc:1034] running on GCE node
W20260807 05:06:14.209223 13069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:14.209780 13063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:06:14.210664 13063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:06:14.211889 13063 hybrid_clock.cc:648] HybridClock initialized: now 1786079174211861 us; error 46 us; skew 500 ppm
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:06:14.215790 13063 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:06:14.217039 13063 webserver.cc:460] Webserver started at http://127.1.17.194:42559/ using document root <none> and password file <none>
I20260807 05:06:14.217814 13063 fs_manager.cc:359] Metadata directory not provided
I20260807 05:06:14.217916 13063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:06:14.218252 13063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:14.240700 13063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "11f02bf37fa149ccbff9a68402a5730d"
format_stamp: "Formatted at 2026-08-07 05:06:14 on dist-test-slave-1x4g"
server_key: "164eba7fa1b88f75b68d3640628135a0"
server_key_iv: "07cb46c331668cee8acbef2c784ed6ec"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:14.241540 13063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "11f02bf37fa149ccbff9a68402a5730d"
format_stamp: "Formatted at 2026-08-07 05:06:14 on dist-test-slave-1x4g"
server_key: "164eba7fa1b88f75b68d3640628135a0"
server_key_iv: "07cb46c331668cee8acbef2c784ed6ec"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:14.261808 13063 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.005s sys 0.000s
I20260807 05:06:14.289872 13079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:14.294045 13063 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260807 05:06:14.294224 13063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "11f02bf37fa149ccbff9a68402a5730d"
format_stamp: "Formatted at 2026-08-07 05:06:14 on dist-test-slave-1x4g"
server_key: "164eba7fa1b88f75b68d3640628135a0"
server_key_iv: "07cb46c331668cee8acbef2c784ed6ec"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:14.294387 13063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:06:14.306470 13063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:06:14.309486 13063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:06:14.309700 13063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:06:14.310395 13063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:06:14.311369 13063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:06:14.311446 13063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:14.311555 13063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:06:14.311614 13063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:14.323979 13063 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:44925
I20260807 05:06:14.324013 13198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:44925 every 8 connection(s)
I20260807 05:06:14.325246 13063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:06:14.327950 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 13063
I20260807 05:06:14.328081 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:06:14.328442 1095 external_mini_cluster.cc:1428] Setting key 3c6490558b92a55f9ca71c6a48ab1f8a
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:06:14.354951 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.327179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:39719 (local address 127.1.17.254:38711)
0807 05:06:14.327320 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:14.327325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:14.329144 (+ 1819us) server_negotiation.cc:407] Connection header received
0807 05:06:14.330392 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:14.330395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:14.330453 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:14.330517 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:06:14.339448 (+ 8931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.340204 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.342150 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.342536 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.342642 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.349795 (+ 7153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:06:14.349800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:06:14.349803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:06:14.349837 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:06:14.351902 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:14.352673 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:14.352677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:14.352679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:14.352818 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:14.353298 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:14.353302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:14.353304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:14.353577 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:06:14.353705 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:14.354552 (+ 847us) server_negotiation.cc:299] Negotiation successful
0807 05:06:14.354718 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:06:14.357195 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.327615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38711 (local address 127.1.17.194:39719)
0807 05:06:14.328981 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:14.329025 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:14.330121 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:14.333224 (+ 3103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:14.333234 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:14.333645 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:06:14.339181 (+ 5536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.339210 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.340441 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.340445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.341981 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.341994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.342884 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.342889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.343105 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.347537 (+ 4432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:06:14.347564 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:06:14.349533 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:06:14.352066 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:06:14.352073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:06:14.352093 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:06:14.352519 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:06:14.352955 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:06:14.352958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:06:14.352960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:06:14.353166 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:06:14.353790 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:06:14.353797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:06:14.354202 (+ 405us) client_negotiation.cc:769] Sending connection context
0807 05:06:14.355558 (+ 1356us) client_negotiation.cc:240] Negotiation successful
0807 05:06:14.355948 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":155,"threads_started":1}
I20260807 05:06:14.358994 13199 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38711
I20260807 05:06:14.359280 13199 heartbeater.cc:464] Registering TS with master...
I20260807 05:06:14.359970 13199 heartbeater.cc:511] Master 127.1.17.254:38711 requested a full tablet report, sending...
I20260807 05:06:14.361361 12835 ts_manager.cc:194] Registered new tserver with Master: 11f02bf37fa149ccbff9a68402a5730d (127.1.17.194:44925)
I20260807 05:06:14.362156 12835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:39719
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:06:14.394590 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:40179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38711
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:06:14.561738 13205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:06:14.562613 13205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:06:14.562764 13205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:06:14.568516 13205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:06:14.568632 13205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:06:14.568755 13205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:06:14.576027 13205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.17.212:40179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:06:14.577905 13205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:06:14.579214 13205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
W20260807 05:06:14.614079 13211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:14.617820 13213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:14.618027 13205 server_base.cc:1034] running on GCE node
W20260807 05:06:14.618697 13210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:14.619251 13205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:06:14.620113 13205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:06:14.623399 13205 hybrid_clock.cc:648] HybridClock initialized: now 1786079174623321 us; error 81 us; skew 500 ppm
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:06:14.627113 13205 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:06:14.628187 13205 webserver.cc:460] Webserver started at http://127.1.17.195:38135/ using document root <none> and password file <none>
I20260807 05:06:14.628937 13205 fs_manager.cc:359] Metadata directory not provided
I20260807 05:06:14.629089 13205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:06:14.629359 13205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:14.656705 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "1862df1ae2cd41aea13dd2bdfb936a2b"
format_stamp: "Formatted at 2026-08-07 05:06:14 on dist-test-slave-1x4g"
server_key: "384c6a3a4be809a0e25d0956854ac1f6"
server_key_iv: "d760f8b930a19a94b3642666340b01db"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:14.657563 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "1862df1ae2cd41aea13dd2bdfb936a2b"
format_stamp: "Formatted at 2026-08-07 05:06:14 on dist-test-slave-1x4g"
server_key: "384c6a3a4be809a0e25d0956854ac1f6"
server_key_iv: "d760f8b930a19a94b3642666340b01db"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:14.672108 13205 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.004s sys 0.000s
I20260807 05:06:14.694218 13221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:14.696148 13205 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:06:14.696293 13205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "1862df1ae2cd41aea13dd2bdfb936a2b"
format_stamp: "Formatted at 2026-08-07 05:06:14 on dist-test-slave-1x4g"
server_key: "384c6a3a4be809a0e25d0956854ac1f6"
server_key_iv: "d760f8b930a19a94b3642666340b01db"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:14.696436 13205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:06:14.717897 13205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:06:14.721650 13205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:06:14.721899 13205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:06:14.722638 13205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:06:14.723834 13205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:06:14.723946 13205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:14.724051 13205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:06:14.724114 13205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:14.736757 13205 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:42967
I20260807 05:06:14.736873 13338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:42967 every 8 connection(s)
I20260807 05:06:14.738065 13205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:06:14.745301 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 13205
I20260807 05:06:14.745441 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:06:14.745712 1095 external_mini_cluster.cc:1428] Setting key 1266401061c2238ac877237caf60ebdc
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:06:14.757395 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.742324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:56409 (local address 127.1.17.254:38711)
0807 05:06:14.742450 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:14.742454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:14.743209 (+ 755us) server_negotiation.cc:407] Connection header received
0807 05:06:14.744496 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:14.744500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:14.744564 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:14.744615 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:06:14.747616 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.748436 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.750122 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.750442 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.750529 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.753485 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:06:14.753488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:06:14.753491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:06:14.753515 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:06:14.755137 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:14.755716 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:14.755720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:14.755722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:14.755801 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:14.756163 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:14.756166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:14.756168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:14.756363 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:06:14.756454 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:14.757112 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 05:06:14.757225 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:06:14.759610 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.741254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38711 (local address 127.1.17.195:56409)
0807 05:06:14.743050 (+ 1796us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:14.743103 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:14.744170 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:14.745049 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:14.746094 (+ 1045us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:14.746508 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:06:14.747455 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.747475 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.748667 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.748670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.749988 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.749999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.750727 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.750730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.750914 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.751591 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:06:14.751610 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:06:14.753298 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:06:14.755269 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:06:14.755274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:06:14.755289 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:06:14.755601 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:06:14.755901 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:06:14.755903 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:06:14.755905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:06:14.756062 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:06:14.756519 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:06:14.756525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:06:14.756844 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 05:06:14.757799 (+ 955us) client_negotiation.cc:240] Negotiation successful
0807 05:06:14.758111 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1572,"thread_start_us":142,"threads_started":1}
I20260807 05:06:14.761061 13339 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38711
I20260807 05:06:14.761279 13339 heartbeater.cc:464] Registering TS with master...
I20260807 05:06:14.761801 13339 heartbeater.cc:511] Master 127.1.17.254:38711 requested a full tablet report, sending...
I20260807 05:06:14.762846 12835 ts_manager.cc:194] Registered new tserver with Master: 1862df1ae2cd41aea13dd2bdfb936a2b (127.1.17.195:42967)
I20260807 05:06:14.763497 12835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:56409
I20260807 05:06:14.771873 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:06:14.784394 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.774403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.1.17.254:38711)
0807 05:06:14.774532 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:14.774536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:14.775158 (+ 622us) server_negotiation.cc:407] Connection header received
0807 05:06:14.775344 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:14.775347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:14.775409 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:14.775465 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:06:14.776429 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.777268 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.778482 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.778820 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.778904 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.780871 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:06:14.780875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:06:14.780878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:06:14.780904 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:06:14.782611 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:14.783095 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:14.783099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:14.783101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:14.783178 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:14.783462 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:14.783465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:14.783466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:14.783666 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:06:14.783763 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:14.784106 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 05:06:14.784222 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:06:14.788455 12835 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:06:14.793058 12835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:06:14.817102 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.804760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:42967 (local address 127.0.0.1:59168)
0807 05:06:14.808420 (+ 3660us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:14.808471 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:14.808595 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:14.810788 (+ 2193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:14.810796 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:14.810817 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:06:14.810940 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.810946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.813159 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.813162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.814496 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.814507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.816523 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.816526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.816659 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.816698 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 05:06:14.816832 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 05:06:14.816950 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3460,"thread_start_us":92,"threads_started":1}
I20260807 05:06:14.817955 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.805923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.1.17.195:42967)
0807 05:06:14.810126 (+ 4203us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:14.810142 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:14.810165 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:06:14.810236 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:14.810241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:14.810458 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:14.810601 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:06:14.811067 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.813034 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.814676 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.816205 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.816307 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.816462 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:14.817711 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0807 05:06:14.817809 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4098,"thread_start_us":147,"threads_started":1}
I20260807 05:06:14.821483 13270 tablet_service.cc:1467] Processing CreateTablet for tablet feab313b29b54cdca9ac83bfd6f56834 (DEFAULT_TABLE table=test-table [id=0a9b24b3777f4d5d9ea674d8c47f2737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:06:14.822464 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.806715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46249 (local address 127.0.0.1:36796)
0807 05:06:14.808826 (+ 2111us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:14.808838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:14.808911 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:14.818345 (+ 9434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:14.818348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:14.818365 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:06:14.818446 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.818451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.819850 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.819853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.821011 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.821022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.822092 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.822096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.822178 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.822202 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:06:14.822250 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 05:06:14.822322 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2027,"thread_start_us":112,"threads_started":1}
I20260807 05:06:14.822710 13270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feab313b29b54cdca9ac83bfd6f56834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:06:14.823180 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.806892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.1.17.193:46249)
0807 05:06:14.817845 (+ 10953us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:14.817852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:14.817886 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:06:14.817965 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:14.817991 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:14.818182 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:14.818596 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:06:14.818730 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.819714 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.821167 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.821940 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.822753 (+ 813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.822898 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:14.822946 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 05:06:14.823026 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10851,"thread_start_us":84,"threads_started":1}
I20260807 05:06:14.825999 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.807894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:44925 (local address 127.0.0.1:39542)
0807 05:06:14.809854 (+ 1960us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:14.809866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:14.809951 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:14.817636 (+ 7685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:14.817639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:14.817656 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:06:14.817745 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.817751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.823250 (+ 5499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.823253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.824452 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:14.824463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.825633 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.825636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:14.825732 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.825754 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:06:14.825798 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 05:06:14.825856 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1905,"thread_start_us":85,"threads_started":1}
I20260807 05:06:14.825769 12987 tablet_service.cc:1467] Processing CreateTablet for tablet feab313b29b54cdca9ac83bfd6f56834 (DEFAULT_TABLE table=test-table [id=0a9b24b3777f4d5d9ea674d8c47f2737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:06:14.826673 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:14.808099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39542 (local address 127.1.17.194:44925)
0807 05:06:14.816990 (+ 8891us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:14.817007 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:14.817036 (+ 29us) server_negotiation.cc:407] Connection header received
0807 05:06:14.817115 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:14.817120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:14.817318 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:14.817486 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:06:14.817887 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.819571 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.824635 (+ 5064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:14.825499 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:14.826266 (+ 767us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:14.826409 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:14.826456 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 05:06:14.826540 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8766,"thread_start_us":115,"threads_started":1}
I20260807 05:06:14.826778 12987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feab313b29b54cdca9ac83bfd6f56834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:06:14.829250 13128 tablet_service.cc:1467] Processing CreateTablet for tablet feab313b29b54cdca9ac83bfd6f56834 (DEFAULT_TABLE table=test-table [id=0a9b24b3777f4d5d9ea674d8c47f2737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:06:14.830240 13128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feab313b29b54cdca9ac83bfd6f56834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:06:14.845209 13359 tablet_bootstrap.cc:492] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058: Bootstrap starting.
I20260807 05:06:14.848790 13359 tablet_bootstrap.cc:654] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058: Neither blocks nor log segments found. Creating new log.
I20260807 05:06:14.849329 13360 tablet_bootstrap.cc:492] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Bootstrap starting.
I20260807 05:06:14.851543 13358 tablet_bootstrap.cc:492] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: Bootstrap starting.
I20260807 05:06:14.852269 13360 tablet_bootstrap.cc:654] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Neither blocks nor log segments found. Creating new log.
I20260807 05:06:14.853299 13359 log.cc:824] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058: Log is configured to *not* fsync() on all Append() calls
I20260807 05:06:14.854429 13358 tablet_bootstrap.cc:654] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: Neither blocks nor log segments found. Creating new log.
I20260807 05:06:14.854626 13360 log.cc:824] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:06:14.855245 13358 log.cc:824] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:06:14.858006 13359 tablet_bootstrap.cc:492] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058: No bootstrap required, opened a new log
I20260807 05:06:14.858278 13359 ts_tablet_manager.cc:1397] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.002s
I20260807 05:06:14.862972 13359 raft_consensus.cc:348] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:14.863440 13359 raft_consensus.cc:374] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:06:14.863572 13359 raft_consensus.cc:729] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1705c2102cdd406eacf5c52446c72058, State: Initialized, Role: FOLLOWER
I20260807 05:06:14.864107 13359 consensus_queue.cc:260] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:14.866410 13359 ts_tablet_manager.cc:1428] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260807 05:06:14.866825 13055 heartbeater.cc:503] Master 127.1.17.254:38711 was elected leader, sending a full tablet report...
I20260807 05:06:14.868722 13360 tablet_bootstrap.cc:492] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: No bootstrap required, opened a new log
I20260807 05:06:14.869010 13360 ts_tablet_manager.cc:1397] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260807 05:06:14.872296 13358 tablet_bootstrap.cc:492] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: No bootstrap required, opened a new log
I20260807 05:06:14.872503 13358 ts_tablet_manager.cc:1397] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260807 05:06:14.873903 13360 raft_consensus.cc:348] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:14.874229 13360 raft_consensus.cc:374] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:06:14.874321 13360 raft_consensus.cc:729] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1862df1ae2cd41aea13dd2bdfb936a2b, State: Initialized, Role: FOLLOWER
I20260807 05:06:14.874903 13360 consensus_queue.cc:260] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:14.875972 13360 ts_tablet_manager.cc:1428] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 05:06:14.876351 13339 heartbeater.cc:503] Master 127.1.17.254:38711 was elected leader, sending a full tablet report...
I20260807 05:06:14.876389 13358 raft_consensus.cc:348] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:14.876644 13358 raft_consensus.cc:374] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:06:14.876695 13358 raft_consensus.cc:729] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f02bf37fa149ccbff9a68402a5730d, State: Initialized, Role: FOLLOWER
I20260807 05:06:14.877115 13358 consensus_queue.cc:260] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:14.878074 13358 ts_tablet_manager.cc:1428] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:06:14.878413 13199 heartbeater.cc:503] Master 127.1.17.254:38711 was elected leader, sending a full tablet report...
Aug 07 05:06:14 dist-test-slave-1x4g krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079174, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
W20260807 05:06:14.944489 13056 tablet.cc:2369] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:06:14.993036 13340 tablet.cc:2369] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:06:15.079326 13200 tablet.cc:2369] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:06:15.081431 13365 raft_consensus.cc:482] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:06:15.081674 13365 raft_consensus.cc:504] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:15.083168 13365 leader_election.cc:288] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1705c2102cdd406eacf5c52446c72058 (127.1.17.193:46249), 11f02bf37fa149ccbff9a68402a5730d (127.1.17.194:44925)
I20260807 05:06:15.089052 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.083656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:44925 (local address 127.1.17.195:42341)
0807 05:06:15.084058 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:15.084076 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:15.084183 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:15.085011 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:15.085014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:15.085035 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:06:15.085165 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:15.085173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.086252 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.086256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:15.087577 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:15.087589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.088622 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.088628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:15.088738 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:15.088761 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:06:15.088841 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:06:15.088908 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310}
I20260807 05:06:15.089480 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.084358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:42341 (local address 127.1.17.194:44925)
0807 05:06:15.084480 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:15.084485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:15.084502 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:06:15.084561 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:15.084564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:15.084614 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:15.084688 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:06:15.085294 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.086101 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.087732 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.088344 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.088479 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:15.088540 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:15.089298 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 05:06:15.089365 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:06:15.090087 13134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feab313b29b54cdca9ac83bfd6f56834" candidate_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f02bf37fa149ccbff9a68402a5730d" is_pre_election: true
I20260807 05:06:15.090438 13134 raft_consensus.cc:2454] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1862df1ae2cd41aea13dd2bdfb936a2b in term 0.
I20260807 05:06:15.090991 13224 leader_election.cc:302] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f02bf37fa149ccbff9a68402a5730d, 1862df1ae2cd41aea13dd2bdfb936a2b; no voters:
I20260807 05:06:15.091300 13365 raft_consensus.cc:2781] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:06:15.091389 13365 raft_consensus.cc:482] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:06:15.091431 13365 raft_consensus.cc:3037] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:06:15.092855 13365 raft_consensus.cc:504] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:15.093254 13365 leader_election.cc:288] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [CANDIDATE]: Term 1 election: Requested vote from peers 1705c2102cdd406eacf5c52446c72058 (127.1.17.193:46249), 11f02bf37fa149ccbff9a68402a5730d (127.1.17.194:44925)
I20260807 05:06:15.093703 13134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feab313b29b54cdca9ac83bfd6f56834" candidate_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f02bf37fa149ccbff9a68402a5730d"
I20260807 05:06:15.093834 13134 raft_consensus.cc:3037] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:06:15.095142 13134 raft_consensus.cc:2454] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1862df1ae2cd41aea13dd2bdfb936a2b in term 1.
I20260807 05:06:15.095479 13224 leader_election.cc:302] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f02bf37fa149ccbff9a68402a5730d, 1862df1ae2cd41aea13dd2bdfb936a2b; no voters:
I20260807 05:06:15.095659 13365 raft_consensus.cc:2781] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:06:15.095877 13365 raft_consensus.cc:686] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 1 LEADER]: Becoming Leader. State: Replica: 1862df1ae2cd41aea13dd2bdfb936a2b, State: Running, Role: LEADER
I20260807 05:06:15.096238 13365 consensus_queue.cc:237] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } }
I20260807 05:06:15.098501 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.085049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:50393 (local address 127.1.17.193:46249)
0807 05:06:15.085165 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:15.085169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:15.088122 (+ 2953us) server_negotiation.cc:407] Connection header received
0807 05:06:15.088233 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:15.088236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:15.088290 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:15.088387 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:06:15.088718 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.089424 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.094406 (+ 4982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.095043 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.095148 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:15.095203 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:15.098269 (+ 3066us) server_negotiation.cc:299] Negotiation successful
0807 05:06:15.098360 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:06:15.099707 12835 catalog_manager.cc:5495] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1862df1ae2cd41aea13dd2bdfb936a2b (127.1.17.195). New cstate: current_term: 1 leader_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 } health_report { overall_health: UNKNOWN } } }
I20260807 05:06:15.101776 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.084621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46249 (local address 127.1.17.195:50393)
0807 05:06:15.088032 (+ 3411us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:15.088046 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:06:15.088143 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:06:15.088433 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:06:15.088438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:06:15.088456 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:06:15.088585 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:15.088594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.089866 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.089870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:15.091317 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:06:15.091330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.096542 (+ 5212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.096546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:06:15.096672 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:15.096699 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 05:06:15.101524 (+ 4825us) client_negotiation.cc:240] Negotiation successful
0807 05:06:15.101612 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3350,"thread_start_us":104,"threads_started":1}
I20260807 05:06:15.102514 13005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feab313b29b54cdca9ac83bfd6f56834" candidate_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1705c2102cdd406eacf5c52446c72058" is_pre_election: true
I20260807 05:06:15.102743 12998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feab313b29b54cdca9ac83bfd6f56834" candidate_uuid: "1862df1ae2cd41aea13dd2bdfb936a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1705c2102cdd406eacf5c52446c72058"
I20260807 05:06:15.102816 13005 raft_consensus.cc:2454] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1862df1ae2cd41aea13dd2bdfb936a2b in term 0.
I20260807 05:06:15.126197 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.121147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59174 (local address 127.1.17.195:42967)
0807 05:06:15.121275 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:15.121280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:15.121296 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:06:15.121378 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:15.121382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:15.121458 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:15.121586 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:06:15.121936 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.122721 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.123972 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:15.124289 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:15.124393 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:15.124853 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:06:15.125428 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:06:15.125581 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:15.126002 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 05:06:15.126061 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:06:15.146013 13134 raft_consensus.cc:1261] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 1 FOLLOWER]: Refusing update from remote peer 1862df1ae2cd41aea13dd2bdfb936a2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:06:15.146790 13370 consensus_queue.cc:1035] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f02bf37fa149ccbff9a68402a5730d" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 44925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:06:15.154882 12998 raft_consensus.cc:3037] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:06:15.156365 12998 raft_consensus.cc:1261] T feab313b29b54cdca9ac83bfd6f56834 P 1705c2102cdd406eacf5c52446c72058 [term 1 FOLLOWER]: Refusing update from remote peer 1862df1ae2cd41aea13dd2bdfb936a2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:06:15.157703 13370 consensus_queue.cc:1035] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1705c2102cdd406eacf5c52446c72058" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:06:15.161684 13373 mvcc.cc:205] Tried to move back new op lower bound from 7315780301368586240 to 7315780301197213696. Current Snapshot: MvccSnapshot[applied={T|T < 7315780301368586240 or (T in {7315780301368586240})}]
I20260807 05:06:15.165354 13374 mvcc.cc:205] Tried to move back new op lower bound from 7315780301368586240 to 7315780301197213696. Current Snapshot: MvccSnapshot[applied={T|T < 7315780301368586240}]
I20260807 05:06:15.167057 13378 mvcc.cc:205] Tried to move back new op lower bound from 7315780301368586240 to 7315780301197213696. Current Snapshot: MvccSnapshot[applied={T|T < 7315780301368586240}]
I20260807 05:06:15.168529 12834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:06:15.168697 12834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:06:15.171392 12834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bba421f53ee347039a608abcec54e495: Sending DeleteTablet for 3 replicas of tablet feab313b29b54cdca9ac83bfd6f56834
I20260807 05:06:15.172351 13128 tablet_service.cc:1514] Processing DeleteTablet for tablet feab313b29b54cdca9ac83bfd6f56834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:06:15 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:39542
I20260807 05:06:15.172642 13270 tablet_service.cc:1514] Processing DeleteTablet for tablet feab313b29b54cdca9ac83bfd6f56834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:06:15 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:59168
I20260807 05:06:15.173122 12987 tablet_service.cc:1514] Processing DeleteTablet for tablet feab313b29b54cdca9ac83bfd6f56834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:06:15 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:36796
I20260807 05:06:15.173347 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 12920
I20260807 05:06:15.175861 13385 tablet_replica.cc:330] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: stopping tablet replica
I20260807 05:06:15.184887 13385 raft_consensus.cc:2229] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:06:15.185312 13385 raft_consensus.cc:2258] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:06:15.185925 13384 tablet_replica.cc:330] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d: stopping tablet replica
I20260807 05:06:15.186257 13384 raft_consensus.cc:2229] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:06:15.186606 13384 raft_consensus.cc:2258] T feab313b29b54cdca9ac83bfd6f56834 P 11f02bf37fa149ccbff9a68402a5730d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:06:15.187850 13385 ts_tablet_manager.cc:1905] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 05:06:15.188867 13225 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:46249 (error 108)
W20260807 05:06:15.189671 12821 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:46249 (error 108)
I20260807 05:06:15.189913 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 13063
I20260807 05:06:15.192544 13385 ts_tablet_manager.cc:1918] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:06:15.193729 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.193331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46249 (local address 127.0.0.1:36798)
0807 05:06:15.193551 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:15.193631 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:46249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260807 05:06:15.194029 12821 catalog_manager.cc:4584] TS 1705c2102cdd406eacf5c52446c72058 (127.1.17.193:46249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet feab313b29b54cdca9ac83bfd6f56834: Network error: Client connection negotiation failed: client connection to 127.1.17.193:46249: connect: Connection refused (error 111)
I20260807 05:06:15.194403 13385 log.cc:1196] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/feab313b29b54cdca9ac83bfd6f56834
I20260807 05:06:15.194856 13385 ts_tablet_manager.cc:1939] T feab313b29b54cdca9ac83bfd6f56834 P 1862df1ae2cd41aea13dd2bdfb936a2b: Deleting consensus metadata
I20260807 05:06:15.196543 12820 catalog_manager.cc:4841] TS 1862df1ae2cd41aea13dd2bdfb936a2b (127.1.17.195:42967): tablet feab313b29b54cdca9ac83bfd6f56834 (table test-table [id=0a9b24b3777f4d5d9ea674d8c47f2737]) successfully deleted
I20260807 05:06:15.197659 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.197366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46249 (local address 127.1.17.195:57811)
0807 05:06:15.197505 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:15.197580 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:46249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 05:06:15.216950 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:15.216469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:44925 (local address 127.0.0.1:39558)
0807 05:06:15.216730 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 05:06:15.216807 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.194:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260807 05:06:15.217564 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 13205
I20260807 05:06:15.224452 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 12801
I20260807 05:06:15.232393 1095 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 05:06:15.576974 1095 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 05:06:15.577134 1095 mini_ranger.cc:66] Stopping Ranger...
I20260807 05:06:15.921376 1095 mini_ranger.cc:68] Stopped Ranger
2026-08-07 05:06:15.921 UTC [12375] LOG: received smart shutdown request
2026-08-07 05:06:15.925 UTC [12375] LOG: background worker "logical replication launcher" (PID 12384) exited with exit code 1
2026-08-07 05:06:15.926 UTC [12379] LOG: shutting down
2026-08-07 05:06:16.078 UTC [12375] LOG: database system is shut down
2026-08-07T05:06:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46263 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:06:16 dist-test-slave-1x4g krb5kdc[13404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:06:16 dist-test-slave-1x4g krb5kdc[13404](info): set up 2 sockets
Aug 07 05:06:16 dist-test-slave-1x4g krb5kdc[13404](info): commencing operation
krb5kdc: starting...
W20260807 05:06:17.242156 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:06:17 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:06:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:06:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/rangeradmin_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/rangeradmin_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/rangerlookup_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/rangerlookup_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
I20260807 05:06:17.410909 1095 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXAtKix/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/postgres -l logfile start
2026-08-07 05:06:19.747 UTC [13434] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 05:06:19.747 UTC [13434] LOG: listening on IPv4 address "127.1.17.212", port 37617
2026-08-07 05:06:19.750 UTC [13434] LOG: listening on Unix socket "/tmp/.s.PGSQL.37617"
2026-08-07 05:06:19.772 UTC [13437] LOG: database system was shut down at 2026-08-07 05:06:18 UTC
2026-08-07 05:06:19.775 UTC [13434] LOG: database system is ready to accept connections
I20260807 05:06:20.730536 1095 mini_postgres.cc:96] Postgres bound to 37617
2026-08-07 05:06:20.738 UTC [13445] FATAL: database "slave" does not exist
127.1.17.212:37617 - accepting connections
I20260807 05:06:20.738682 1095 mini_ranger.cc:161] Starting Ranger...
I20260807 05:06:20.754328 1095 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 05:06:21.196538 1095 mini_ranger.cc:87] Created ranger Postgres database
I20260807 05:06:21.196647 1095 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-admin
2026-08-07 05:06:21,462 [I] DB FLAVOR :POSTGRES
2026-08-07 05:06:21,463 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:06:21,463 [I] Checking connection..
2026-08-07 05:06:21,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 05:06:21,692 [I] Checking connection passed.
2026-08-07 05:06:21,692 [I] --------- Verifying version history table ---------
2026-08-07 05:06:21,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:06:21,919 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:06:21,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:06:22,186 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 05:06:22,187 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 05:06:22,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 05:06:22.448 UTC [13511] WARNING: there is no transaction in progress
2026-08-07 05:06:22,455 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 05:06:22,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 05:06:22,702 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 05:06:22,702 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:06:22,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 05:06:22,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 05:06:23,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-07 05:06:23,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:06:23,651 [I] Table x_portal_user does not exist in database ranger
2026-08-07 05:06:23,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:06:23,875 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 05:06:23,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:06:24,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:06:24,361 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 05:06:24,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 05:06:25.296 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.310 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.326 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.340 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.355 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.371 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.421 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.427 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.440 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.449 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.459 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.472 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.481 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.487 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.492 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:25.498 UTC [13619] WARNING: there is no transaction in progress
2026-08-07 05:06:26,086 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 05:06:26,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 05:06:26,330 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 05:06:26,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 05:06:26,591 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 05:06:26,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 05:06:26,850 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 05:06:26,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 05:06:27,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 05:06:27,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-07 05:06:27,587 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 05:06:27,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 05:06:27,844 [I] DB_PATCHES have already been applied
I20260807 05:06:27.850576 1095 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXAtKix/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 05:06:27.850647 1095 mini_ranger.cc:193] Using host: 127.1.17.212
I20260807 05:06:27.855358 1095 mini_ranger.cc:227] Ranger admin URL: http://127.1.17.212:57679
Aug 07, 2026 5:06:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:06:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:06:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 5:06:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 5:06:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 5:06:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 5:06:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 05:06:28.968713 1095 mini_ranger.cc:160] Time spent starting Ranger: real 8.230s user 0.007s sys 0.004s
Aug 07, 2026 5:06:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 5:06:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 5:06:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 5:06:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 5:06:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 05:06:50.640861 1095 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.17.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.17.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/rangerkms_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.17.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/rangerkms_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.17.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.17.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.17.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/HTTP_127.1.17.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 05:06:50 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079210, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 05:06:50.744590 1095 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 05:06:50.756619 1095 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 05:06:51.582693 1095 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 05:06:51.582789 1095 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms
I20260807 05:06:51.582808 1095 mini_ranger_kms.cc:227] Using postgres at 127.1.17.212:37617
2026-08-07 05:06:51,802 [I] DB FLAVOR :POSTGRES
2026-08-07 05:06:51,802 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 05:06:51,802 [I] Checking connection
2026-08-07 05:06:51,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 05:06:52,023 [I] connection success
2026-08-07 05:06:52,023 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 05:06:52,024 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 05:06:52,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 05:06:52,259 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 05:06:52,259 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 05:06:52,259 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 05:06:52,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.17.212:37617/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 05:06:52,531 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 05:06:52.998257 1095 mini_ranger_kms.cc:310] Created kms service
I20260807 05:06:53.207933 1095 mini_ranger_kms.cc:326] Created kudu user
I20260807 05:06:53.251716 1095 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 05:06:53 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079210, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:53.649241 1095 mini_ranger_kms.cc:384] Added ranger policy
I20260807 05:06:53.649400 1095 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXAtKix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXAtKix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 05:06:53.649425 1095 mini_ranger_kms.cc:240] Using host: 127.1.17.212
I20260807 05:06:53.654171 1095 mini_ranger_kms.cc:276] Ranger KMS PID: 13799
I20260807 05:06:53.654260 1095 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.17.212:33979
Aug 07, 2026 5:06:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 5:06:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 5:06:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 5:06:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 5:06:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 5:06:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 05:06:55 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079215, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.17.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:06:55.922267 1095 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.178s user 0.006s sys 0.198s
I20260807 05:06:55.922363 1095 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 05:06:55.922444 1095 mini_ranger_kms.cc:401] 127.1.17.212:33979/kms/v1/keys
Aug 07 05:06:56 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079215, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.17.212@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
Aug 07 05:06:56 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079215, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.17.212@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
Aug 07 05:06:58 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:06:58.864718 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34027
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:34027
--ranger_config_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:06:58.973474 13850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:06:58.973795 13850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:06:58.973868 13850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:06:58.977929 13850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:06:58.978010 13850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:06:58.978047 13850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:06:58.978104 13850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:06:58.978142 13850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:06:58.983244 13850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.17.212:33979/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:34027
--ranger_config_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:06:58.984243 13850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:06:58.985296 13850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:06:58.993957 13856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:58.993973 13859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:58.994069 13850 server_base.cc:1034] running on GCE node
W20260807 05:06:58.993999 13857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:58.994678 13850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:06:58.995666 13850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:06:58.996856 13850 hybrid_clock.cc:648] HybridClock initialized: now 1786079218996841 us; error 35 us; skew 500 ppm
Aug 07 05:06:58 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079218, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:06:59.003953 13850 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:06:59.004942 13850 webserver.cc:460] Webserver started at http://127.1.17.254:34089/ using document root <none> and password file <none>
I20260807 05:06:59.005520 13850 fs_manager.cc:359] Metadata directory not provided
I20260807 05:06:59.005602 13850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:06:59.005829 13850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:06:59 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079218, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:59.063222 13850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "e4097adb6bf348c29201360e9ddd9461"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "d6ad25aa59cac21fb2ee65f88cdd9a7a"
server_key_iv: "7aea93ffff0555f01d12b5b657581ed4"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:59.064005 13850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "e4097adb6bf348c29201360e9ddd9461"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "d6ad25aa59cac21fb2ee65f88cdd9a7a"
server_key_iv: "7aea93ffff0555f01d12b5b657581ed4"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:59.069085 13850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 05:06:59.086194 13866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:06:59.087587 13850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 05:06:59.087771 13850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "e4097adb6bf348c29201360e9ddd9461"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "d6ad25aa59cac21fb2ee65f88cdd9a7a"
server_key_iv: "7aea93ffff0555f01d12b5b657581ed4"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:59.087911 13850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:06:59.099676 13850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:06:59.102607 13850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:06:59.102815 13850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:06:59.110970 13850 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:34027
I20260807 05:06:59.111034 13919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:34027 every 8 connection(s)
I20260807 05:06:59.111964 13850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:06:59.113255 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 13850
I20260807 05:06:59.113374 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:06:59.113672 1095 external_mini_cluster.cc:1428] Setting key fc870f8073e0e83598c44fd2a6f7b050
I20260807 05:06:59.115513 13920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 05:06:59 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:06:59.122622 13920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Bootstrap starting.
I20260807 05:06:59.125321 13920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Neither blocks nor log segments found. Creating new log.
I20260807 05:06:59.126679 13920 log.cc:824] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Log is configured to *not* fsync() on all Append() calls
I20260807 05:06:59.127648 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:06:59.115165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.1.17.254:34027)
0807 05:06:59.115607 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 05:06:59.115629 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:06:59.115678 (+ 49us) server_negotiation.cc:407] Connection header received
0807 05:06:59.116310 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:06:59.116326 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:06:59.116666 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:06:59.117060 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:06:59.117877 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:59.118852 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:59.119716 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:06:59.120177 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:06:59.120341 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:06:59.122911 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:06:59.122917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:06:59.122931 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:06:59.122956 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:06:59.124621 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:59.125127 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:59.125133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:59.125137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:59.125240 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:06:59.125527 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:06:59.125532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:06:59.125533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:06:59.125920 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:06:59.126140 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 05:06:59.126443 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 05:06:59.126752 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":142,"threads_started":1}
I20260807 05:06:59.128877 13920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: No bootstrap required, opened a new log
I20260807 05:06:59.132798 13920 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:06:59.133081 13920 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:06:59.133157 13920 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4097adb6bf348c29201360e9ddd9461, State: Initialized, Role: FOLLOWER
I20260807 05:06:59.133653 13920 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:06:59.133816 13920 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:06:59.133890 13920 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:06:59.133989 13920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:06:59.135079 13920 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:06:59.135483 13920 leader_election.cc:302] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4097adb6bf348c29201360e9ddd9461; no voters:
I20260807 05:06:59.135819 13920 leader_election.cc:288] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:06:59.135921 13925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:06:59.136206 13925 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 LEADER]: Becoming Leader. State: Replica: e4097adb6bf348c29201360e9ddd9461, State: Running, Role: LEADER
I20260807 05:06:59.136554 13925 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:06:59.137249 13920 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:06:59.138159 13927 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4097adb6bf348c29201360e9ddd9461. Latest consensus state: current_term: 1 leader_uuid: "e4097adb6bf348c29201360e9ddd9461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } } }
I20260807 05:06:59.138278 13927 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: This master's current role is: LEADER
I20260807 05:06:59.138615 13926 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4097adb6bf348c29201360e9ddd9461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } } }
I20260807 05:06:59.138690 13926 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: This master's current role is: LEADER
I20260807 05:06:59.139688 13934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:06:59.139955 13920 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 05:06:59.145699 13934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:06:59.153147 13934 catalog_manager.cc:1347] Generated new cluster ID: dc43660f149844a08376575d9bae8f1d
I20260807 05:06:59.153213 13934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:06:59.173367 13934 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:06:59.174620 13934 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:06:59.188633 13934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Generated new TSK 0
I20260807 05:06:59.189499 13934 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:06:59.827816 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34027
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:06:59.945675 13966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:06:59.945945 13966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:06:59.946017 13966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:06:59.950588 13966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:06:59.950666 13966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:06:59.950750 13966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:06:59.957767 13966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.17.212:33979/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:06:59.959056 13966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:06:59.960139 13966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:06:59.970427 13974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:59.970544 13977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:06:59.970400 13973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:06:59.971159 13966 server_base.cc:1034] running on GCE node
I20260807 05:06:59.971653 13966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:06:59.972325 13966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:06:59.975634 13966 hybrid_clock.cc:648] HybridClock initialized: now 1786079219975592 us; error 49 us; skew 500 ppm
Aug 07 05:06:59 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:06:59.978756 13966 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:06:59.979713 13966 webserver.cc:460] Webserver started at http://127.1.17.193:45709/ using document root <none> and password file <none>
I20260807 05:06:59.980302 13966 fs_manager.cc:359] Metadata directory not provided
I20260807 05:06:59.980361 13966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:06:59.980583 13966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:06:59 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:06:59.995674 13966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "970cea052b2240d28cc84995098c36a9"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "a771e5490a0b53184719350ecb0c5eb4"
server_key_iv: "32209f310b883ec77bfca353c9288365"
server_key_version: "kuduclusterkey@0"
I20260807 05:06:59.996544 13966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "970cea052b2240d28cc84995098c36a9"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "a771e5490a0b53184719350ecb0c5eb4"
server_key_iv: "32209f310b883ec77bfca353c9288365"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.006155 13966 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260807 05:07:00.025396 13984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.027323 13966 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 05:07:00.027537 13966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "970cea052b2240d28cc84995098c36a9"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "a771e5490a0b53184719350ecb0c5eb4"
server_key_iv: "32209f310b883ec77bfca353c9288365"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.027762 13966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:00.053104 13966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:00.057181 13966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:00.057430 13966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:00.058215 13966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:00.059310 13966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:00.059422 13966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.059559 13966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:00.059645 13966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.071143 13966 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:38797
I20260807 05:07:00.071172 14104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:38797 every 8 connection(s)
I20260807 05:07:00.072296 13966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:07:00.076414 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 13966
I20260807 05:07:00.076545 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:07:00.076929 1095 external_mini_cluster.cc:1428] Setting key 8d5bcf63202179326d331f24e126749e
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:00.088668 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:00.074361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:55965 (local address 127.1.17.254:34027)
0807 05:07:00.074657 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:00.074666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:00.075495 (+ 829us) server_negotiation.cc:407] Connection header received
0807 05:07:00.076689 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:00.076694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:00.076760 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:00.076854 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:00.078726 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.079539 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.081394 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.081762 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.081844 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:00.084534 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:00.084538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:00.084540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:00.084571 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:00.086454 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:00.087077 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:00.087082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:00.087084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:00.087182 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:00.087523 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:00.087528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:00.087529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:00.087727 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:00.087840 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:00.088315 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 05:07:00.088470 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":111,"threads_started":1}
I20260807 05:07:00.089702 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:00.074863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34027 (local address 127.1.17.193:55965)
0807 05:07:00.075335 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:00.075389 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:00.076400 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:00.077291 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:00.077301 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:00.077869 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:00.078557 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:00.078578 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.079708 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.079714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:00.081233 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:00.081248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.082078 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.082084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:00.082289 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:00.082982 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:00.082999 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:00.084358 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:00.086588 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:00.086594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:00.086610 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:00.086966 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:00.087276 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:00.087280 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:00.087282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:00.087424 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:00.087831 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:00.087838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:00.088087 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 05:07:00.088275 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 05:07:00.088601 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":134,"threads_started":1}
I20260807 05:07:00.091141 14105 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34027
I20260807 05:07:00.091459 14105 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:00.092067 14105 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:00.093806 13884 ts_manager.cc:194] Registered new tserver with Master: 970cea052b2240d28cc84995098c36a9 (127.1.17.193:38797)
I20260807 05:07:00.094942 13884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:55965
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:07:00.127774 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34027
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 05:07:00.261544 14111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:00.261821 14111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:00.261950 14111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:00.266172 14111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:00.266245 14111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:00.266358 14111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:07:00.271828 14111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.17.212:33979/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:00.272953 14111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:00.273995 14111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:00.283828 14120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:00.283937 14118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:00.283828 14117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:00.284973 14111 server_base.cc:1034] running on GCE node
I20260807 05:07:00.285386 14111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:00.286051 14111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:00.289321 14111 hybrid_clock.cc:648] HybridClock initialized: now 1786079220289290 us; error 39 us; skew 500 ppm
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:00.292351 14111 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:07:00.293344 14111 webserver.cc:460] Webserver started at http://127.1.17.194:38733/ using document root <none> and password file <none>
I20260807 05:07:00.293926 14111 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:00.294024 14111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:00.294298 14111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:00.308609 14111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "35c0d7c91ea14c978c9117485f7d5056"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "04d5a51270dcb9e85d5a8b6da76545d5"
server_key_iv: "d42e609e9cadcbc8f9f052d1a54ffb84"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.309810 14111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "35c0d7c91ea14c978c9117485f7d5056"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "04d5a51270dcb9e85d5a8b6da76545d5"
server_key_iv: "d42e609e9cadcbc8f9f052d1a54ffb84"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.317332 14111 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260807 05:07:00.336912 14127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.339339 14111 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.003s
I20260807 05:07:00.339546 14111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "35c0d7c91ea14c978c9117485f7d5056"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "04d5a51270dcb9e85d5a8b6da76545d5"
server_key_iv: "d42e609e9cadcbc8f9f052d1a54ffb84"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.339766 14111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:00.355453 14111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:00.359382 14111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:00.359608 14111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:00.360352 14111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:00.361479 14111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:00.361593 14111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.361688 14111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:00.361749 14111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.372166 14111 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:40765
I20260807 05:07:00.373380 14111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:07:00.377151 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 14111
I20260807 05:07:00.377274 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:07:00.377622 1095 external_mini_cluster.cc:1428] Setting key 2eff8f385af693c27770a1478d4f6fff
I20260807 05:07:00.385457 14245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:40765 every 8 connection(s)
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:00.412293 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:00.390485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:44103 (local address 127.1.17.254:34027)
0807 05:07:00.390610 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:00.390615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:00.391108 (+ 493us) server_negotiation.cc:407] Connection header received
0807 05:07:00.392457 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:00.392461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:00.392533 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:00.392589 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:00.395930 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.396867 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.401226 (+ 4359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.401595 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.401853 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:00.407794 (+ 5941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:00.407799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:00.407802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:00.407832 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:00.409669 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:00.410334 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:00.410338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:00.410340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:00.410422 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:00.410862 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:00.410866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:00.410868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:00.411117 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:00.411202 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:00.411948 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 05:07:00.412096 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 05:07:00.414143 14249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:00.389352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34027 (local address 127.1.17.194:44103)
0807 05:07:00.390933 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:00.391002 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:00.392200 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:00.393099 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:00.393109 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:00.393469 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:00.394139 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:00.394157 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.399229 (+ 5072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.399232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:00.401072 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:00.401085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.402093 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.402097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:00.405016 (+ 2919us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:00.405776 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:00.405798 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:00.407558 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:00.409816 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:00.409823 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:00.409841 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:00.410210 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:00.410550 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:00.410553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:00.410555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:00.410739 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:00.411273 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:00.411280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:00.411633 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 05:07:00.412578 (+ 945us) client_negotiation.cc:240] Negotiation successful
0807 05:07:00.412981 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1344,"thread_start_us":154,"threads_started":1}
I20260807 05:07:00.415621 14246 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34027
I20260807 05:07:00.415855 14246 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:00.416409 14246 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:00.417763 13884 ts_manager.cc:194] Registered new tserver with Master: 35c0d7c91ea14c978c9117485f7d5056 (127.1.17.194:40765)
I20260807 05:07:00.418671 13884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:44103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:07:00.431862 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34027
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:00.610852 14251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:00.611255 14251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:00.611366 14251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:00.617311 14251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:00.617491 14251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:00.617620 14251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:07:00.625306 14251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.17.212:33979/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:00.626863 14251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:00.628239 14251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:00.641409 14258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:00.642474 14260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:00.642832 14257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:00.643213 14251 server_base.cc:1034] running on GCE node
I20260807 05:07:00.643661 14251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:00.644469 14251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:00.648782 14251 hybrid_clock.cc:648] HybridClock initialized: now 1786079220648463 us; error 326 us; skew 500 ppm
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:00.654486 14251 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:07:00.655735 14251 webserver.cc:460] Webserver started at http://127.1.17.195:32835/ using document root <none> and password file <none>
I20260807 05:07:00.656543 14251 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:00.656670 14251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:00.656986 14251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:00.676420 14251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "dee0976229a5ecbb5f053ec18cb073ed"
server_key_iv: "85859341476780c50a11fcae55201a70"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.677228 14251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "dee0976229a5ecbb5f053ec18cb073ed"
server_key_iv: "85859341476780c50a11fcae55201a70"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.683624 14251 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 05:07:00.705250 14267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:00.710564 14251 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.003s
I20260807 05:07:00.710717 14251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "dee0976229a5ecbb5f053ec18cb073ed"
server_key_iv: "85859341476780c50a11fcae55201a70"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:00.710870 14251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:00.744122 14251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:00.748095 14251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:00.748303 14251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:00.749027 14251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:00.750113 14251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:00.750165 14251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.750221 14251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:00.750243 14251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:00.764636 14251 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:38961
I20260807 05:07:00.765872 14251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:07:00.767424 14385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:38961 every 8 connection(s)
I20260807 05:07:00.774992 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 14251
I20260807 05:07:00.775113 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:07:00.775398 1095 external_mini_cluster.cc:1428] Setting key f4cabd48038fc691752f14eba69a59c7
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:00.785924 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:00.769315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:40747 (local address 127.1.17.254:34027)
0807 05:07:00.769447 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:00.769452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:00.771633 (+ 2181us) server_negotiation.cc:407] Connection header received
0807 05:07:00.773029 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:00.773033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:00.773086 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:00.773145 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:00.774842 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.775778 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.777667 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.778061 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.778137 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:00.781506 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:00.781510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:00.781514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:00.781542 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:00.783412 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:00.784073 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:00.784077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:00.784079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:00.784164 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:00.784564 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:00.784567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:00.784569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:00.784816 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:00.784914 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:00.785601 (+ 687us) server_negotiation.cc:299] Negotiation successful
0807 05:07:00.785729 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:07:00.787837 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:00.769854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34027 (local address 127.1.17.195:40747)
0807 05:07:00.771469 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:00.771527 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:00.772667 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:00.773580 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:00.773589 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:00.773976 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:00.774660 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:00.774680 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.775962 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.775965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:00.777520 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:00.777531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:00.778348 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:00.778351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:00.778562 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:00.779321 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:00.779343 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:00.781304 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:00.783563 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:00.783569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:00.783587 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:00.783950 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:00.784271 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:00.784274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:00.784277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:00.784452 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:00.784986 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:00.784993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:00.785311 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 05:07:00.786387 (+ 1076us) client_negotiation.cc:240] Negotiation successful
0807 05:07:00.786744 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1374,"thread_start_us":137,"threads_started":1}
I20260807 05:07:00.789410 14386 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34027
I20260807 05:07:00.789651 14386 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:00.790223 14386 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:00.791417 13884 ts_manager.cc:194] Registered new tserver with Master: b38e3dd941e94b68ba3eb95f4f4a69d3 (127.1.17.195:38961)
I20260807 05:07:00.792225 13884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:40747
I20260807 05:07:00.801450 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:07:00.802407 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 13966
I20260807 05:07:00.811851 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 14111
I20260807 05:07:00.820448 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 14251
I20260807 05:07:00.830830 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 13850
I20260807 05:07:00.841368 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34027
--webserver_interface=127.1.17.254
--webserver_port=34089
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:34027
--ranger_config_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:00.953415 14389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:00.953637 14389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:00.953672 14389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:00.958233 14389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:00.958288 14389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:00.958305 14389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:00.958320 14389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:00.958360 14389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:00.963388 14389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:34027
--ranger_config_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=34089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:00.964676 14389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:00.965767 14389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:00.974216 14395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:00.974236 14397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:00.974323 14389 server_base.cc:1034] running on GCE node
W20260807 05:07:00.974232 14394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:00.975028 14389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:00.976125 14389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:00.977316 14389 hybrid_clock.cc:648] HybridClock initialized: now 1786079220977300 us; error 29 us; skew 500 ppm
Aug 07 05:07:00 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:00.980507 14389 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:00.981415 14389 webserver.cc:460] Webserver started at http://127.1.17.254:34089/ using document root <none> and password file <none>
I20260807 05:07:00.981923 14389 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:00.981997 14389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:01.031811 14389 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:07:01.034377 14406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.035702 14389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:07:01.035833 14389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "e4097adb6bf348c29201360e9ddd9461"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "d6ad25aa59cac21fb2ee65f88cdd9a7a"
server_key_iv: "7aea93ffff0555f01d12b5b657581ed4"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:01.036379 14389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:01.053660 14389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:01.056349 14389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:01.056555 14389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:01.064018 14389 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:34027
I20260807 05:07:01.064024 14459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:34027 every 8 connection(s)
I20260807 05:07:01.065174 14389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:01.069080 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 14389
I20260807 05:07:01.069520 14460 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:07:01.069577 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:38797
--local_ip_for_outbound_sockets=127.1.17.193
--tserver_master_addrs=127.1.17.254:34027
--webserver_port=45709
--webserver_interface=127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:01.071942 14460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Bootstrap starting.
I20260807 05:07:01.088325 14460 log.cc:824] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:01.094789 14460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:07:01.095194 14460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Bootstrap complete.
I20260807 05:07:01.099282 14460 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:07:01.099658 14460 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4097adb6bf348c29201360e9ddd9461, State: Initialized, Role: FOLLOWER
I20260807 05:07:01.100160 14460 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:07:01.100272 14460 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:07:01.100350 14460 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:07:01.100441 14460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:07:01.102258 14460 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:07:01.102584 14460 leader_election.cc:302] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4097adb6bf348c29201360e9ddd9461; no voters:
I20260807 05:07:01.102902 14460 leader_election.cc:288] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:07:01.103025 14465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:07:01.103243 14465 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [term 2 LEADER]: Becoming Leader. State: Replica: e4097adb6bf348c29201360e9ddd9461, State: Running, Role: LEADER
I20260807 05:07:01.103554 14465 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } }
I20260807 05:07:01.104097 14460 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:01.104956 14466 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e4097adb6bf348c29201360e9ddd9461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } } }
I20260807 05:07:01.105391 14466 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:01.105181 14467 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4097adb6bf348c29201360e9ddd9461. Latest consensus state: current_term: 2 leader_uuid: "e4097adb6bf348c29201360e9ddd9461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4097adb6bf348c29201360e9ddd9461" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34027 } } }
I20260807 05:07:01.105661 14467 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:01.105870 14474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:01.109015 14474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:01.109577 14460 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 05:07:01.109786 14474 catalog_manager.cc:1259] Loaded cluster ID: dc43660f149844a08376575d9bae8f1d
I20260807 05:07:01.109879 14474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:07:01.112233 14474 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:07:01.112911 14474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Loaded TSK: 0
I20260807 05:07:01.114094 14474 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:07:01.213812 14463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:01.214144 14463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:01.214206 14463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:01.220507 14463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:01.220577 14463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:01.220654 14463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:07:01.228236 14463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:38797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=45709
--tserver_master_addrs=127.1.17.254:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:01.229509 14463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:01.230830 14463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:01.243119 14499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:01.243534 14501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:01.243584 14498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:01.244721 14463 server_base.cc:1034] running on GCE node
I20260807 05:07:01.245183 14463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:01.245970 14463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:01.247169 14463 hybrid_clock.cc:648] HybridClock initialized: now 1786079221247147 us; error 37 us; skew 500 ppm
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:01.250852 14463 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:07:01.252087 14463 webserver.cc:460] Webserver started at http://127.1.17.193:45709/ using document root <none> and password file <none>
I20260807 05:07:01.252959 14463 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:01.253077 14463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:01.276850 14463 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:07:01.280328 14509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.281772 14463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:07:01.281925 14463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "970cea052b2240d28cc84995098c36a9"
format_stamp: "Formatted at 2026-08-07 05:06:59 on dist-test-slave-1x4g"
server_key: "a771e5490a0b53184719350ecb0c5eb4"
server_key_iv: "32209f310b883ec77bfca353c9288365"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:01.282519 14463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:01.295513 14463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:01.298810 14463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:01.299016 14463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:01.299700 14463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:01.300712 14463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:01.300791 14463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.300848 14463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:01.300875 14463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.312391 14463 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:38797
I20260807 05:07:01.314734 14463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:07:01.315727 14627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:38797 every 8 connection(s)
I20260807 05:07:01.319206 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 14463
I20260807 05:07:01.319702 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:40765
--local_ip_for_outbound_sockets=127.1.17.194
--tserver_master_addrs=127.1.17.254:34027
--webserver_port=38733
--webserver_interface=127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:01.349490 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:01.318148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34027 (local address 127.1.17.193:39945)
0807 05:07:01.321428 (+ 3280us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:01.321486 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:01.322551 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:01.324144 (+ 1593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:01.324153 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:01.324619 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:01.325347 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:01.325370 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.327401 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.327405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:01.328845 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:01.328856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.330185 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.330190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:01.330385 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:01.331160 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:01.331182 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:01.342076 (+ 10894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:01.345168 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:01.345176 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:01.345193 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:01.345612 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:01.346088 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:01.346091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:01.346094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:01.346296 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:01.347262 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:01.347269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:01.347610 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 05:07:01.347826 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 05:07:01.348223 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3064,"thread_start_us":145,"threads_started":1}
I20260807 05:07:01.357306 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:01.317990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:39945 (local address 127.1.17.254:34027)
0807 05:07:01.321122 (+ 3132us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:01.321146 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:01.322872 (+ 1726us) server_negotiation.cc:407] Connection header received
0807 05:07:01.323231 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:01.323248 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:01.323661 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:01.324030 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:01.325572 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.327231 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.329002 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.329766 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.329959 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:01.342420 (+ 12461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:01.342428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:01.342454 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:01.342495 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:01.344977 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.345772 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.345779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.345786 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.345962 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.346442 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.346446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.346448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.347033 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:01.347182 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:01.355757 (+ 8575us) server_negotiation.cc:299] Negotiation successful
0807 05:07:01.356158 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2898,"thread_start_us":112,"threads_started":1}
I20260807 05:07:01.360355 14628 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34027
I20260807 05:07:01.360612 14628 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.361243 14628 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
W20260807 05:07:01.363484 14628 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:34027 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 05:07:01.363600 14628 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.363757 14628 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:01.364547 14628 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.364713 14628 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
W20260807 05:07:01.365456 14628 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:07:01.468315 14632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:01.468560 14632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:01.468600 14632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:01.472841 14632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:01.472891 14632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:01.472949 14632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:07:01.478999 14632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:40765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=38733
--tserver_master_addrs=127.1.17.254:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:01.480170 14632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:01.481377 14632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:01.491523 14637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:01.491575 14632 server_base.cc:1034] running on GCE node
W20260807 05:07:01.491535 14640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:01.491884 14638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:01.492329 14632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:01.493054 14632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:01.494241 14632 hybrid_clock.cc:648] HybridClock initialized: now 1786079221494204 us; error 56 us; skew 500 ppm
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:01.497465 14632 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:07:01.498409 14632 webserver.cc:460] Webserver started at http://127.1.17.194:38733/ using document root <none> and password file <none>
I20260807 05:07:01.499028 14632 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:01.499127 14632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:01.518975 14632 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:07:01.522743 14649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.524194 14632 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:07:01.524336 14632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "35c0d7c91ea14c978c9117485f7d5056"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "04d5a51270dcb9e85d5a8b6da76545d5"
server_key_iv: "d42e609e9cadcbc8f9f052d1a54ffb84"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:01.524981 14632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:01.548327 14632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:01.551903 14632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:01.552124 14632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:01.552752 14632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:01.553735 14632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:01.553807 14632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.553875 14632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:01.553929 14632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.565347 14632 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:40765
I20260807 05:07:01.565493 14767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:40765 every 8 connection(s)
I20260807 05:07:01.566646 14632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:07:01.573618 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 14632
I20260807 05:07:01.574141 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.17.212:33979/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:38961
--local_ip_for_outbound_sockets=127.1.17.195
--tserver_master_addrs=127.1.17.254:34027
--webserver_port=32835
--webserver_interface=127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:01.584254 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:01.569291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34027 (local address 127.1.17.194:43405)
0807 05:07:01.569790 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:01.569842 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:01.570808 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:01.571464 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:01.571471 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:01.571796 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:01.572431 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:01.572448 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.573320 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.573323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:01.575026 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:01.575041 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.575715 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.575722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:01.575971 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:01.576709 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:01.576733 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:01.578547 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:01.580807 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:01.580816 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:01.580834 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:01.581193 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:01.581601 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:01.581605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:01.581607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:01.581786 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:01.582372 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:01.582379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:01.582696 (+ 317us) client_negotiation.cc:769] Sending connection context
0807 05:07:01.582907 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 05:07:01.583239 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":172,"threads_started":1}
I20260807 05:07:01.585047 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:01.568814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:43405 (local address 127.1.17.254:34027)
0807 05:07:01.568994 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:01.568999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:01.569965 (+ 966us) server_negotiation.cc:407] Connection header received
0807 05:07:01.571062 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:01.571066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:01.571147 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:01.571253 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:01.572581 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.573190 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.575176 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.575525 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.575661 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:01.578732 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:01.578736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:01.578739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:01.578767 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:01.580639 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.581321 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.581327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.581330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.581448 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.581915 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.581918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.581920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.582169 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:01.582269 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:01.584730 (+ 2461us) server_negotiation.cc:299] Negotiation successful
0807 05:07:01.584884 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:07:01.586755 14768 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34027
I20260807 05:07:01.587003 14768 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.587567 14768 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
W20260807 05:07:01.589591 14768 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:34027 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 05:07:01.589722 14768 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.589879 14768 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:01.590791 14768 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.591075 14768 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
W20260807 05:07:01.591838 14768 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:07:01.728369 14772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:01.728626 14772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:01.728665 14772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:01.732683 14772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:01.732736 14772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:01.732857 14772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:07:01.738385 14772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.17.212:33979/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:38961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=32835
--tserver_master_addrs=127.1.17.254:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:01.739729 14772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:01.741048 14772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:01.752144 14778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:01.752398 14781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:01.753118 14772 server_base.cc:1034] running on GCE node
W20260807 05:07:01.753289 14779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:01.753737 14772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:01.754415 14772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:01.757702 14772 hybrid_clock.cc:648] HybridClock initialized: now 1786079221757662 us; error 49 us; skew 500 ppm
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:01.761848 14772 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:07:01.762980 14772 webserver.cc:460] Webserver started at http://127.1.17.195:32835/ using document root <none> and password file <none>
I20260807 05:07:01.763706 14772 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:01.763803 14772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:01.785926 14772 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 05:07:01.789664 14790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.792034 14772 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 05:07:01.792176 14772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3"
format_stamp: "Formatted at 2026-08-07 05:07:00 on dist-test-slave-1x4g"
server_key: "dee0976229a5ecbb5f053ec18cb073ed"
server_key_iv: "85859341476780c50a11fcae55201a70"
server_key_version: "kuduclusterkey@0"
I20260807 05:07:01.792953 14772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:01.809149 14772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:01.814278 14772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:01.814504 14772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:01.815194 14772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:01.816293 14772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:01.816362 14772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.816418 14772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:01.816452 14772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:01.827926 14772 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:38961
I20260807 05:07:01.828125 14907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:38961 every 8 connection(s)
I20260807 05:07:01.829231 14772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:07:01.835213 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 14772
Aug 07 05:07:01 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079221, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:01.849138 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:01.832876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:35261 (local address 127.1.17.254:34027)
0807 05:07:01.833036 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:01.833041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:01.834891 (+ 1850us) server_negotiation.cc:407] Connection header received
0807 05:07:01.836151 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:01.836155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:01.836220 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:01.836280 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:01.838093 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.838915 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.840735 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.841106 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.841191 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:01.844603 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:01.844607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:01.844610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:01.844641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:01.846486 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.847137 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.847141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.847142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.847237 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.847726 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.847729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.847731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.847956 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:01.848033 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:01.848826 (+ 793us) server_negotiation.cc:299] Negotiation successful
0807 05:07:01.848955 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:07:01.851576 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:01.833389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34027 (local address 127.1.17.195:35261)
0807 05:07:01.834694 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:01.834753 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:01.835876 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:01.836757 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:01.836767 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:01.837170 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:01.837899 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:01.837921 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.839128 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.839131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:01.840589 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:01.840602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.841419 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.841424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:01.841630 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:01.842502 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:01.842524 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:01.844383 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:01.846628 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:01.846636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:01.846654 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:01.847010 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:01.847366 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:01.847372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:01.847375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:01.847602 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:01.848120 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:01.848128 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:01.848487 (+ 359us) client_negotiation.cc:769] Sending connection context
0807 05:07:01.848724 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 05:07:01.849110 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"thread_start_us":153,"threads_started":1}
I20260807 05:07:01.852964 14908 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34027
I20260807 05:07:01.853241 14908 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.853858 14908 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
W20260807 05:07:01.855628 14908 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:34027 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 05:07:01.855785 14908 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.855968 14908 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:01.856889 14908 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:01.857079 14908 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
W20260807 05:07:01.857824 14908 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 05:07:01.861624 14912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:01.836285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.1.17.254:34027)
0807 05:07:01.836560 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:01.836569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:01.845811 (+ 9242us) server_negotiation.cc:407] Connection header received
0807 05:07:01.845860 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:01.845863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:01.845914 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:01.846009 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:01.850498 (+ 4489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.851334 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.854849 (+ 3515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:01.855169 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:01.855250 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:01.857990 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:01.857994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:01.857997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:01.858022 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:01.859831 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.860347 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.860350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.860351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.860424 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:01.860741 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:01.860744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:01.860745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:01.860971 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:01.861119 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:01.861367 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 05:07:01.861475 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":96,"threads_started":1}
Aug 07 05:07:02 dist-test-slave-1x4g krb5kdc[13404](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786079222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:02.365700 14628 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:02.366046 14628 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:02.367352 14424 ts_manager.cc:194] Registered new tserver with Master: 970cea052b2240d28cc84995098c36a9 (127.1.17.193:38797)
I20260807 05:07:02.369087 14424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:39945
I20260807 05:07:02.592106 14768 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:02.592423 14768 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:02.593199 14424 ts_manager.cc:194] Registered new tserver with Master: 35c0d7c91ea14c978c9117485f7d5056 (127.1.17.194:40765)
I20260807 05:07:02.594034 14424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:43405
Aug 07 05:07:02 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
Aug 07 05:07:02 dist-test-slave-1x4g krb5kdc[13404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for HTTP/127.1.17.212@KRBTEST.COM
I20260807 05:07:02.858071 14908 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:02.858399 14908 heartbeater.cc:511] Master 127.1.17.254:34027 requested a full tablet report, sending...
I20260807 05:07:02.859247 14424 ts_manager.cc:194] Registered new tserver with Master: b38e3dd941e94b68ba3eb95f4f4a69d3 (127.1.17.195:38961)
I20260807 05:07:02.860061 14424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:35261
I20260807 05:07:02.864620 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:07:02.882663 14931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:02.866441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.1.17.254:34027)
0807 05:07:02.869125 (+ 2684us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:02.869130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:02.869704 (+ 574us) server_negotiation.cc:407] Connection header received
0807 05:07:02.869919 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:02.869923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:02.869979 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:02.870067 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:02.873173 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.874281 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.875498 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.875861 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.875965 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:02.877886 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:02.877890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:02.877893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:02.877920 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:02.879949 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:02.880556 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:02.880560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:02.880561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:02.880660 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:02.881095 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:02.881100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:02.881101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:02.881362 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:02.881511 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:02.882256 (+ 745us) server_negotiation.cc:299] Negotiation successful
0807 05:07:02.882403 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2595,"thread_start_us":94,"threads_started":1}
I20260807 05:07:02.887187 14424 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:07:02.892341 14424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:07:02.927726 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:02.913721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38797 (local address 127.0.0.1:43386)
0807 05:07:02.916293 (+ 2572us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:02.916305 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:02.916381 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:02.917378 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:02.917385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:02.917410 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:02.917546 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:02.917554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.919130 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.919133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:02.920560 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:02.920572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.927139 (+ 6567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.927143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:02.927265 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:02.927306 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 05:07:02.927420 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 05:07:02.927543 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2498,"thread_start_us":103,"threads_started":1}
I20260807 05:07:02.928685 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:02.914552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.1.17.193:38797)
0807 05:07:02.916641 (+ 2089us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:02.916648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:02.916691 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:07:02.916772 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:02.916777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:02.917033 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:02.917194 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:02.917693 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.918984 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.925915 (+ 6931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.926792 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.926901 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:02.927068 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:02.928413 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0807 05:07:02.928525 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1950,"thread_start_us":97,"threads_started":1}
I20260807 05:07:02.932819 14942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:02.913669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:40765 (local address 127.0.0.1:37528)
0807 05:07:02.915822 (+ 2153us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:02.915876 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:02.916022 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:02.921563 (+ 5541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:02.921566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:02.921584 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:02.921677 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:02.921684 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.929107 (+ 7423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.929110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:02.930307 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:02.930318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.932419 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.932423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:02.932521 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:02.932547 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 05:07:02.932596 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 05:07:02.932662 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1972,"thread_start_us":119,"threads_started":1}
I20260807 05:07:02.933745 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:02.914592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.1.17.194:40765)
0807 05:07:02.921097 (+ 6505us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:02.921103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:02.921128 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:07:02.921205 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:02.921211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:02.921406 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:02.921845 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:02.924571 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.925785 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.930469 (+ 4684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.931341 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.933286 (+ 1945us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:02.933442 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:02.933494 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 05:07:02.933589 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6148,"thread_start_us":107,"threads_started":1}
I20260807 05:07:02.936733 14679 tablet_service.cc:1467] Processing CreateTablet for tablet 3cecf0174cf24ff4925967b852fafc20 (DEFAULT_TABLE table=test-table [id=92d7d0721d53476bb2969d4ff6bd4775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:02.937886 14679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cecf0174cf24ff4925967b852fafc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:02.934867 14542 tablet_service.cc:1467] Processing CreateTablet for tablet 3cecf0174cf24ff4925967b852fafc20 (DEFAULT_TABLE table=test-table [id=92d7d0721d53476bb2969d4ff6bd4775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:02.948346 14945 tablet_bootstrap.cc:492] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Bootstrap starting.
I20260807 05:07:02.950887 14945 tablet_bootstrap.cc:654] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:02.951867 14945 log.cc:824] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:02.954209 14945 tablet_bootstrap.cc:492] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: No bootstrap required, opened a new log
I20260807 05:07:02.954254 14542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cecf0174cf24ff4925967b852fafc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:02.954407 14945 ts_tablet_manager.cc:1397] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:07:02.959198 14945 raft_consensus.cc:348] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:02.959638 14945 raft_consensus.cc:374] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:02.959751 14945 raft_consensus.cc:729] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c0d7c91ea14c978c9117485f7d5056, State: Initialized, Role: FOLLOWER
I20260807 05:07:02.960383 14945 consensus_queue.cc:260] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:02.961592 14945 ts_tablet_manager.cc:1428] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 05:07:02.962019 14768 heartbeater.cc:503] Master 127.1.17.254:34027 was elected leader, sending a full tablet report...
I20260807 05:07:02.967892 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:02.915320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:38961 (local address 127.0.0.1:34430)
0807 05:07:02.925978 (+ 10658us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:02.925994 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:02.926112 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:02.927960 (+ 1848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:02.927965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:02.927992 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:02.928310 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:02.928323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.961312 (+ 32989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.961317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:02.963701 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:02.963718 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.965253 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.965258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:02.965378 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:02.965409 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 05:07:02.965505 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 05:07:02.967704 (+ 2199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10594,"thread_start_us":116,"threads_started":1}
I20260807 05:07:02.973302 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:02.915494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.1.17.195:38961)
0807 05:07:02.920993 (+ 5499us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:02.921000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:02.926223 (+ 5223us) server_negotiation.cc:407] Connection header received
0807 05:07:02.926312 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:02.926317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:02.926524 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:02.926676 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:02.928648 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.961109 (+ 32461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.963909 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:02.964892 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:02.965014 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:02.965179 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:02.972622 (+ 7443us) server_negotiation.cc:299] Negotiation successful
0807 05:07:02.973127 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5381,"thread_start_us":56,"threads_started":1}
I20260807 05:07:02.973850 14948 tablet_bootstrap.cc:492] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9: Bootstrap starting.
I20260807 05:07:02.976692 14838 tablet_service.cc:1467] Processing CreateTablet for tablet 3cecf0174cf24ff4925967b852fafc20 (DEFAULT_TABLE table=test-table [id=92d7d0721d53476bb2969d4ff6bd4775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:02.977614 14948 tablet_bootstrap.cc:654] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:02.977921 14838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cecf0174cf24ff4925967b852fafc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:02.978622 14948 log.cc:824] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:02.980913 14948 tablet_bootstrap.cc:492] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9: No bootstrap required, opened a new log
I20260807 05:07:02.981125 14948 ts_tablet_manager.cc:1397] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 05:07:02.985739 14948 raft_consensus.cc:348] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:02.986100 14948 raft_consensus.cc:374] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:02.986200 14948 raft_consensus.cc:729] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970cea052b2240d28cc84995098c36a9, State: Initialized, Role: FOLLOWER
I20260807 05:07:02.986795 14948 consensus_queue.cc:260] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:02.988015 14948 ts_tablet_manager.cc:1428] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 05:07:02.988417 14628 heartbeater.cc:503] Master 127.1.17.254:34027 was elected leader, sending a full tablet report...
I20260807 05:07:03.006515 14951 tablet_bootstrap.cc:492] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Bootstrap starting.
I20260807 05:07:03.016043 14951 tablet_bootstrap.cc:654] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:03.017146 14951 log.cc:824] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:03.027496 14951 tablet_bootstrap.cc:492] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: No bootstrap required, opened a new log
I20260807 05:07:03.027750 14951 ts_tablet_manager.cc:1397] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.000s
I20260807 05:07:03.032274 14951 raft_consensus.cc:348] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:03.032620 14951 raft_consensus.cc:374] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:03.032680 14951 raft_consensus.cc:729] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38e3dd941e94b68ba3eb95f4f4a69d3, State: Initialized, Role: FOLLOWER
I20260807 05:07:03.038582 14951 consensus_queue.cc:260] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:03.039975 14951 ts_tablet_manager.cc:1428] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260807 05:07:03.040436 14908 heartbeater.cc:503] Master 127.1.17.254:34027 was elected leader, sending a full tablet report...
W20260807 05:07:03.066996 14629 tablet.cc:2369] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:07:03.071961 14769 tablet.cc:2369] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:07:03.080971 14909 tablet.cc:2369] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:07:03.081688 14953 raft_consensus.cc:482] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:03.081907 14953 raft_consensus.cc:504] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:03.083312 14953 leader_election.cc:288] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35c0d7c91ea14c978c9117485f7d5056 (127.1.17.194:40765), 970cea052b2240d28cc84995098c36a9 (127.1.17.193:38797)
I20260807 05:07:03.093245 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:03.083719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:53589 (local address 127.1.17.194:40765)
0807 05:07:03.084096 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:03.084101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:03.084115 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:07:03.084177 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:03.084180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:03.084249 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:03.084338 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:03.084702 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.085561 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.091693 (+ 6132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.092322 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.092407 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:03.092460 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:03.093047 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 05:07:03.093126 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297}
I20260807 05:07:03.094771 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:03.083718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:36859 (local address 127.1.17.193:38797)
0807 05:07:03.083988 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:03.083993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:03.087825 (+ 3832us) server_negotiation.cc:407] Connection header received
0807 05:07:03.090972 (+ 3147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:03.090976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:03.091029 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:03.091078 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:03.091483 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.092204 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.093506 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.094063 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.094153 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:03.094202 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:03.094581 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 05:07:03.094653 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260807 05:07:03.094975 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:03.083603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:40765 (local address 127.1.17.195:53589)
0807 05:07:03.083941 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:03.083958 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:03.084067 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:03.084379 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:03.084382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:03.084403 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:03.084558 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:03.084565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.087069 (+ 2504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.087073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:03.091567 (+ 4494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:03.091576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.092815 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.092818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:03.092916 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:03.092937 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:07:03.093744 (+ 807us) client_negotiation.cc:240] Negotiation successful
0807 05:07:03.093816 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":140,"threads_started":1}
I20260807 05:07:03.095152 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:03.083603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38797 (local address 127.1.17.195:36859)
0807 05:07:03.087736 (+ 4133us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:03.087750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:03.090823 (+ 3073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:03.091207 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:03.091210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:03.091226 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:03.091332 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:03.091338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.092334 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.092337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:03.093366 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:03.093378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.094314 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.094317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:03.094431 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:03.094453 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:07:03.094926 (+ 473us) client_negotiation.cc:240] Negotiation successful
0807 05:07:03.095002 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4051,"mutex_wait_us":59,"thread_start_us":101,"threads_started":1}
I20260807 05:07:03.095616 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cecf0174cf24ff4925967b852fafc20" candidate_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c0d7c91ea14c978c9117485f7d5056" is_pre_election: true
I20260807 05:07:03.095674 14580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cecf0174cf24ff4925967b852fafc20" candidate_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970cea052b2240d28cc84995098c36a9" is_pre_election: true
I20260807 05:07:03.095939 14710 raft_consensus.cc:2454] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38e3dd941e94b68ba3eb95f4f4a69d3 in term 0.
I20260807 05:07:03.095939 14580 raft_consensus.cc:2454] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38e3dd941e94b68ba3eb95f4f4a69d3 in term 0.
I20260807 05:07:03.096400 14792 leader_election.cc:302] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c0d7c91ea14c978c9117485f7d5056, b38e3dd941e94b68ba3eb95f4f4a69d3; no voters:
I20260807 05:07:03.096925 14953 raft_consensus.cc:2781] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:07:03.097018 14953 raft_consensus.cc:482] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:07:03.097066 14953 raft_consensus.cc:3037] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:03.098387 14953 raft_consensus.cc:504] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:03.098716 14953 leader_election.cc:288] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [CANDIDATE]: Term 1 election: Requested vote from peers 35c0d7c91ea14c978c9117485f7d5056 (127.1.17.194:40765), 970cea052b2240d28cc84995098c36a9 (127.1.17.193:38797)
I20260807 05:07:03.099081 14580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cecf0174cf24ff4925967b852fafc20" candidate_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970cea052b2240d28cc84995098c36a9"
I20260807 05:07:03.099210 14580 raft_consensus.cc:3037] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:03.100370 14580 raft_consensus.cc:2454] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38e3dd941e94b68ba3eb95f4f4a69d3 in term 1.
I20260807 05:07:03.100658 14793 leader_election.cc:302] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 970cea052b2240d28cc84995098c36a9, b38e3dd941e94b68ba3eb95f4f4a69d3; no voters:
I20260807 05:07:03.101145 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cecf0174cf24ff4925967b852fafc20" candidate_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c0d7c91ea14c978c9117485f7d5056"
I20260807 05:07:03.101266 14710 raft_consensus.cc:3037] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:03.102437 14710 raft_consensus.cc:2454] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38e3dd941e94b68ba3eb95f4f4a69d3 in term 1.
I20260807 05:07:03.102800 14953 raft_consensus.cc:2781] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:03.103026 14953 raft_consensus.cc:686] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 1 LEADER]: Becoming Leader. State: Replica: b38e3dd941e94b68ba3eb95f4f4a69d3, State: Running, Role: LEADER
I20260807 05:07:03.103394 14953 consensus_queue.cc:237] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } }
I20260807 05:07:03.106755 14424 catalog_manager.cc:5495] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b38e3dd941e94b68ba3eb95f4f4a69d3 (127.1.17.195). New cstate: current_term: 1 leader_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b38e3dd941e94b68ba3eb95f4f4a69d3" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 38961 } health_report { overall_health: HEALTHY } } }
I20260807 05:07:03.122479 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:03.117727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34432 (local address 127.1.17.195:38961)
0807 05:07:03.117838 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:03.117842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:03.117854 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:07:03.117936 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:03.117938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:03.117995 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:03.118092 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:07:03.118366 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.119119 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.120363 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:03.120700 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:03.120846 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:03.121285 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:07:03.121784 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:07:03.121908 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:03.122284 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 05:07:03.122338 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:07:03.129269 14580 raft_consensus.cc:1261] T 3cecf0174cf24ff4925967b852fafc20 P 970cea052b2240d28cc84995098c36a9 [term 1 FOLLOWER]: Refusing update from remote peer b38e3dd941e94b68ba3eb95f4f4a69d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:03.130715 14710 raft_consensus.cc:1261] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 1 FOLLOWER]: Refusing update from remote peer b38e3dd941e94b68ba3eb95f4f4a69d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:03.131313 14953 consensus_queue.cc:1035] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "970cea052b2240d28cc84995098c36a9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:03.131755 14953 consensus_queue.cc:1035] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c0d7c91ea14c978c9117485f7d5056" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 40765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:03.149034 14959 mvcc.cc:205] Tried to move back new op lower bound from 7315780497929728000 to 7315780497834266624. Current Snapshot: MvccSnapshot[applied={T|T < 7315780497929728000}]
I20260807 05:07:03.153389 14960 mvcc.cc:205] Tried to move back new op lower bound from 7315780497929728000 to 7315780497834266624. Current Snapshot: MvccSnapshot[applied={T|T < 7315780497929728000}]
I20260807 05:07:03.150305 14962 mvcc.cc:205] Tried to move back new op lower bound from 7315780497929728000 to 7315780497834266624. Current Snapshot: MvccSnapshot[applied={T|T < 7315780497929728000 or (T in {7315780497929728000})}]
I20260807 05:07:03.164403 14424 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:03.164590 14424 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:03.167339 14424 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4097adb6bf348c29201360e9ddd9461: Sending DeleteTablet for 3 replicas of tablet 3cecf0174cf24ff4925967b852fafc20
I20260807 05:07:03.168375 14679 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cecf0174cf24ff4925967b852fafc20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:03 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:37528
I20260807 05:07:03.169288 14542 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cecf0174cf24ff4925967b852fafc20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:03 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:43386
I20260807 05:07:03.169519 14838 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cecf0174cf24ff4925967b852fafc20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:03 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:34430
I20260807 05:07:03.170445 14969 tablet_replica.cc:330] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: stopping tablet replica
I20260807 05:07:03.170497 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 14463
I20260807 05:07:03.172130 14971 tablet_replica.cc:330] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: stopping tablet replica
I20260807 05:07:03.172457 14971 raft_consensus.cc:2229] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:07:03.172952 14971 raft_consensus.cc:2258] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:03.173388 14969 raft_consensus.cc:2229] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:07:03.173925 14969 raft_consensus.cc:2258] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:03.175631 14969 ts_tablet_manager.cc:1905] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:03.177960 14971 ts_tablet_manager.cc:1905] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:03.180757 14971 ts_tablet_manager.cc:1918] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:03.180876 14971 log.cc:1196] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/3cecf0174cf24ff4925967b852fafc20
I20260807 05:07:03.181191 14971 ts_tablet_manager.cc:1939] T 3cecf0174cf24ff4925967b852fafc20 P b38e3dd941e94b68ba3eb95f4f4a69d3: Deleting consensus metadata
I20260807 05:07:03.182367 14408 catalog_manager.cc:4841] TS b38e3dd941e94b68ba3eb95f4f4a69d3 (127.1.17.195:38961): tablet 3cecf0174cf24ff4925967b852fafc20 (table test-table [id=92d7d0721d53476bb2969d4ff6bd4775]) successfully deleted
I20260807 05:07:03.186582 14969 ts_tablet_manager.cc:1918] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:03.188963 14969 log.cc:1196] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/3cecf0174cf24ff4925967b852fafc20
I20260807 05:07:03.189338 14969 ts_tablet_manager.cc:1939] T 3cecf0174cf24ff4925967b852fafc20 P 35c0d7c91ea14c978c9117485f7d5056: Deleting consensus metadata
I20260807 05:07:03.190636 14409 catalog_manager.cc:4841] TS 35c0d7c91ea14c978c9117485f7d5056 (127.1.17.194:40765): tablet 3cecf0174cf24ff4925967b852fafc20 (table test-table [id=92d7d0721d53476bb2969d4ff6bd4775]) successfully deleted
W20260807 05:07:03.201195 14410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:38797 (error 108)
I20260807 05:07:03.202837 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:03.202490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38797 (local address 127.0.0.1:43388)
0807 05:07:03.202676 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:03.202749 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260807 05:07:03.203027 14410 catalog_manager.cc:4584] TS 970cea052b2240d28cc84995098c36a9 (127.1.17.193:38797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3cecf0174cf24ff4925967b852fafc20: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38797: connect: Connection refused (error 111)
I20260807 05:07:03.203359 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 14632
I20260807 05:07:03.214154 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 14772
I20260807 05:07:03.224519 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 14389
I20260807 05:07:03.234395 1095 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 05:07:03.578429 1095 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 05:07:03.578536 1095 mini_ranger.cc:66] Stopping Ranger...
I20260807 05:07:03.923539 1095 mini_ranger.cc:68] Stopped Ranger
2026-08-07 05:07:03.923 UTC [13434] LOG: received smart shutdown request
2026-08-07 05:07:03.928 UTC [13434] LOG: background worker "logical replication launcher" (PID 13443) exited with exit code 1
2026-08-07 05:07:03.929 UTC [13438] LOG: shutting down
2026-08-07 05:07:04.079 UTC [13434] LOG: database system is shut down
2026-08-07T05:07:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48002 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:07:04 dist-test-slave-1x4g krb5kdc[14988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:07:04 dist-test-slave-1x4g krb5kdc[14988](info): set up 2 sockets
Aug 07 05:07:04 dist-test-slave-1x4g krb5kdc[14988](info): commencing operation
krb5kdc: starting...
W20260807 05:07:05.253355 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:07:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:07:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:07:05.379571 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43849
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43849,127.1.17.253:41145,127.1.17.252:41541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:05.490851 15001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:05.491163 15001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:05.491248 15001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:05.495306 15001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:05.495400 15001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:05.495436 15001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:05.495477 15001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:05.495510 15001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:05.500687 15001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43849,127.1.17.253:41145,127.1.17.252:41541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:05.501799 15001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:05.502854 15001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:05.511464 15006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:05.511503 15001 server_base.cc:1034] running on GCE node
W20260807 05:07:05.511479 15007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:05.511476 15009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:05.512331 15001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:05.513486 15001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:05.514655 15001 hybrid_clock.cc:648] HybridClock initialized: now 1786079225514635 us; error 30 us; skew 500 ppm
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:05.517638 15001 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:05.518536 15001 webserver.cc:460] Webserver started at http://127.1.17.254:39627/ using document root <none> and password file <none>
I20260807 05:07:05.519099 15001 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:05.519162 15001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:05.519407 15001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:05.521557 15001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "a8fd5c6895e24e17a52cfc30f3440a2d"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "5c675aed2d1fab6fa341506ecf1acd86"
server_key_iv: "e8528d3d0e0b8f5cce1afd32385b00db"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.522133 15001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "a8fd5c6895e24e17a52cfc30f3440a2d"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "5c675aed2d1fab6fa341506ecf1acd86"
server_key_iv: "e8528d3d0e0b8f5cce1afd32385b00db"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.525892 15001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:07:05.528267 15017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:05.529542 15001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:05.529685 15001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "a8fd5c6895e24e17a52cfc30f3440a2d"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "5c675aed2d1fab6fa341506ecf1acd86"
server_key_iv: "e8528d3d0e0b8f5cce1afd32385b00db"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.529795 15001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:05.541604 15001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:05.545122 15001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:05.545351 15001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:05.553087 15001 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:43849
I20260807 05:07:05.553141 15070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:43849 every 8 connection(s)
I20260807 05:07:05.554134 15001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:05.556003 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15001
I20260807 05:07:05.556137 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:07:05.556444 1095 external_mini_cluster.cc:1428] Setting key 764d70c707358145896b7a44e530e7ac
I20260807 05:07:05.557343 15071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:05.561512 15071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.17.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.253@KRBTEST.COM" created.
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:05.578094 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.563782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43849 (local address 127.0.0.1:51616)
0807 05:07:05.564264 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.564318 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:05.565238 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:05.566033 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:05.566038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:05.566588 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:05.566878 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.566914 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.568125 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.568131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.569267 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.569283 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.569949 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.569956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.570122 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.570819 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:05.570835 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:05.572562 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:05.575086 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.575094 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.575103 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.575339 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.575716 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.575718 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.575719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.575823 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.576612 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:05.576618 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:05.576741 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 05:07:05.576957 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 05:07:05.577225 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":138,"threads_started":1}
I20260807 05:07:05.578091 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.563783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.1.17.254:43849)
0807 05:07:05.564392 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:05.564405 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:05.564452 (+ 47us) server_negotiation.cc:407] Connection header received
0807 05:07:05.565592 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:05.565601 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:05.565813 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:05.565972 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:05.567064 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.567966 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.569411 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.569782 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.569905 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.572727 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:05.572737 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:05.572754 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:05.572830 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:05.574956 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.575459 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.575468 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.575472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.575617 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.575918 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.575923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.575927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.576474 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:05.576637 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:05.577070 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 05:07:05.577332 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":468,"spinlock_wait_cycles":5376,"thread_start_us":186,"threads_started":1}
I20260807 05:07:05.580687 15071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:07:05.581535 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.581254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35210)
0807 05:07:05.581393 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.581476 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 05:07:05.581738 15018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
I20260807 05:07:05.582609 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.582402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35222)
0807 05:07:05.582515 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.582559 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 05:07:05.582762 15071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:41145: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab.
Entry for principal kudu/127.1.17.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab.
I20260807 05:07:05.587361 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:41145
--webserver_interface=127.1.17.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab
--principal=kudu/127.1.17.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43849,127.1.17.253:41145,127.1.17.252:41541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:05.632014 15071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } attempt: 1
I20260807 05:07:05.632934 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.632598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35232)
0807 05:07:05.632743 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.632847 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 05:07:05.633741 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.633538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35234)
0807 05:07:05.633638 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.633686 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 05:07:05.633898 15071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:41145: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
W20260807 05:07:05.700744 15078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:05.701010 15078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:05.701078 15078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 05:07:05.702090 15071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } attempt: 2
I20260807 05:07:05.702940 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.702645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35242)
0807 05:07:05.702796 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.702858 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 05:07:05.703652 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.703444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35254)
0807 05:07:05.703546 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.703601 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 05:07:05.703898 15071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:41145: Network error: Client connection negotiation failed: client connection to 127.1.17.253:41145: connect: Connection refused (error 111)
W20260807 05:07:05.705859 15078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:05.705912 15078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:05.705933 15078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:05.705989 15078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:05.706022 15078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:05.711427 15078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43849,127.1.17.253:41145,127.1.17.252:41541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab
--principal=kudu/127.1.17.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:41145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.17.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:05.712469 15078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:05.713567 15078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:05.725335 15084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:05.725481 15078 server_base.cc:1034] running on GCE node
W20260807 05:07:05.725335 15085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:05.725713 15087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:05.726401 15078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:05.727854 15078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:05.729059 15078 hybrid_clock.cc:648] HybridClock initialized: now 1786079225729040 us; error 67 us; skew 500 ppm
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:05.732020 15078 init.cc:377] Logged in from keytab as kudu/127.1.17.253@KRBTEST.COM (short username kudu)
I20260807 05:07:05.733011 15078 webserver.cc:460] Webserver started at http://127.1.17.253:41915/ using document root <none> and password file <none>
I20260807 05:07:05.733695 15078 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:05.733769 15078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:05.734050 15078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:05.736171 15078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data/instance:
uuid: "4b5b890976cf4ab989fd217c1faca793"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "97f6945dbc4f10b2c69ac21f698cd142"
server_key_iv: "1e36015c508c0fda98d5228783d85d04"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.736863 15078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/wal/instance:
uuid: "4b5b890976cf4ab989fd217c1faca793"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "97f6945dbc4f10b2c69ac21f698cd142"
server_key_iv: "1e36015c508c0fda98d5228783d85d04"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.740681 15078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260807 05:07:05.743294 15094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:05.744634 15078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:05.744858 15078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/wal
uuid: "4b5b890976cf4ab989fd217c1faca793"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "97f6945dbc4f10b2c69ac21f698cd142"
server_key_iv: "1e36015c508c0fda98d5228783d85d04"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.744980 15078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:05.756183 15078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:05.759219 15078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:05.759418 15078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:05.766825 15078 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.253:41145
I20260807 05:07:05.766901 15146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.253:41145 every 8 connection(s)
I20260807 05:07:05.767823 15078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
I20260807 05:07:05.770586 15147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:05.774067 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15078
I20260807 05:07:05.774196 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-1/wal/instance
I20260807 05:07:05.774487 1095 external_mini_cluster.cc:1428] Setting key bddcbe7796653a98ecb0e83543a6fb68
I20260807 05:07:05.774307 15147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.17.252@KRBTEST.COM; defaulting to no policy
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.252@KRBTEST.COM" created.
I20260807 05:07:05.788447 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.776244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.1.17.254:43849)
0807 05:07:05.776388 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:05.776396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:05.777374 (+ 978us) server_negotiation.cc:407] Connection header received
0807 05:07:05.778292 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:05.778295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:05.778347 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:05.778483 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:05.779635 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.780180 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.781380 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.781618 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.781734 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.784328 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:05.784331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:05.784333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:05.784360 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:05.785849 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.786402 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.786409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.786414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.786524 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.786993 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.786997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.787000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.787283 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:05.787416 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:05.787913 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 05:07:05.788114 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:07:05.789072 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.776704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43849 (local address 127.0.0.1:51624)
0807 05:07:05.777204 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.777255 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:05.778098 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:05.778657 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:05.778664 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:05.779014 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:05.779482 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.779495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.780326 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.780329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.781245 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.781255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.781783 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.781787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.781948 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.782576 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:05.782590 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:05.784176 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:05.785980 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.785985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.785996 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.786238 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.786657 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.786663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.786668 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.786854 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.787408 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:05.787414 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:05.787654 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 05:07:05.787808 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 05:07:05.788049 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
I20260807 05:07:05.790535 15147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:07:05.795095 15071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } attempt: 3
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
I20260807 05:07:05.802707 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.791015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35270)
0807 05:07:05.791128 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.791143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:05.791212 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:05.791824 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:05.791826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:05.792003 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:05.792075 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.792080 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.793093 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.793100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.793626 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.793634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.794258 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.794262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.794351 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.794860 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:05.794871 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:05.798246 (+ 3375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:05.801097 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.801099 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.801101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.801321 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.801645 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.801647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.801648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.801720 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.802172 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:05.802175 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:05.802294 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 05:07:05.802396 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 05:07:05.802525 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 05:07:05.802815 15153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.791170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35270 (local address 127.1.17.253:41145)
0807 05:07:05.791508 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:05.791517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:05.791535 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:07:05.791568 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:05.791572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:05.791708 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:05.791846 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:05.792182 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.792976 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.793743 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.794144 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.794262 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.798387 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:05.798396 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:05.798407 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:05.798427 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:05.800986 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.801445 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.801451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.801453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.801550 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.801812 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.801817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.801818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.802060 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:05.802200 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:05.802491 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 05:07:05.802634 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":232,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab.
Entry for principal kudu/127.1.17.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab.
I20260807 05:07:05.804557 15147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:07:05.804628 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:41541
--webserver_interface=127.1.17.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab
--principal=kudu/127.1.17.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43849,127.1.17.253:41145,127.1.17.252:41541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:05.805270 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.805029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57774)
0807 05:07:05.805136 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.805215 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 05:07:05.805462 15095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
I20260807 05:07:05.805500 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.795528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35274)
0807 05:07:05.795662 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.795677 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:05.795749 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:05.796273 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:05.796276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:05.796446 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:05.797077 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.797087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.797907 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.797921 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.798603 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:05.798612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.799310 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.799314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:05.799393 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.799855 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:05.799871 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:05.801325 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:05.803504 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.803510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.803515 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.803782 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.804135 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:05.804138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:05.804140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:05.804240 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:05.804820 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:05.804827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:05.804949 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 05:07:05.805054 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 05:07:05.805218 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 05:07:05.806340 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.795604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.1.17.253:41145)
0807 05:07:05.796003 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:05.796010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:05.796021 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:07:05.796048 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:05.796050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:05.796078 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:05.796264 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:05.797247 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.797777 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.798764 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:05.799176 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:05.799316 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:05.801461 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:05.801464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:05.801466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:05.801526 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:05.803361 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.803915 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.803918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.803920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.804004 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:05.804387 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:05.804394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:05.804399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:05.804637 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:05.805823 (+ 1186us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:05.805914 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 05:07:05.806052 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":128,"threads_started":1}
I20260807 05:07:05.806872 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.806337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57778)
0807 05:07:05.806727 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.806796 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313}
I20260807 05:07:05.807008 15071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } has no permanent_uuid. Determining permanent_uuid...
W20260807 05:07:05.807067 15147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:41541: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
I20260807 05:07:05.807762 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.807484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57790)
0807 05:07:05.807625 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.807702 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 05:07:05.808568 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.808247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57802)
0807 05:07:05.808418 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.808484 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 05:07:05.808719 15071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:41541: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
I20260807 05:07:05.839932 15071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } attempt: 1
I20260807 05:07:05.840999 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.840545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57808)
0807 05:07:05.840726 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.840845 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 05:07:05.841817 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.841551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57810)
0807 05:07:05.841675 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.841739 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 05:07:05.841997 15071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:41541: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
I20260807 05:07:05.856340 15147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } attempt: 1
I20260807 05:07:05.857220 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.856936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57812)
0807 05:07:05.857081 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.857145 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 05:07:05.857933 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.857692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57828)
0807 05:07:05.857804 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.857855 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 05:07:05.858062 15147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:41541: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
I20260807 05:07:05.917203 15071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } attempt: 2
I20260807 05:07:05.918160 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.917744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57830)
0807 05:07:05.917954 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.918054 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 05:07:05.918973 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.918729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57836)
0807 05:07:05.918831 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.918902 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 05:07:05.919122 15071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:41541: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
W20260807 05:07:05.922399 15156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:05.922634 15156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:05.922744 15156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 05:07:05.926220 15147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } attempt: 2
I20260807 05:07:05.926942 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.926680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57844)
0807 05:07:05.926828 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.926880 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 05:07:05.927071 15156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:05.927140 15156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:05.927189 15156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:05.927230 15156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:05.927269 15156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:05.927637 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:05.927427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57858)
0807 05:07:05.927532 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:05.927587 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 05:07:05.927754 15147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:41541: Network error: Client connection negotiation failed: client connection to 127.1.17.252:41541: connect: Connection refused (error 111)
I20260807 05:07:05.932642 15156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43849,127.1.17.253:41145,127.1.17.252:41541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab
--principal=kudu/127.1.17.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:41541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.17.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:05.933866 15156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:05.934952 15156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:05.943686 15161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:05.943641 15162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:05.943956 15156 server_base.cc:1034] running on GCE node
W20260807 05:07:05.944046 15164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:05.944633 15156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:05.945812 15156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:05.947017 15156 hybrid_clock.cc:648] HybridClock initialized: now 1786079225946983 us; error 45 us; skew 500 ppm
Aug 07 05:07:05 dist-test-slave-1x4g krb5kdc[14988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:05.950069 15156 init.cc:377] Logged in from keytab as kudu/127.1.17.252@KRBTEST.COM (short username kudu)
I20260807 05:07:05.951092 15156 webserver.cc:460] Webserver started at http://127.1.17.252:43341/ using document root <none> and password file <none>
I20260807 05:07:05.951670 15156 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:05.951786 15156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:05.952019 15156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:05.954239 15156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data/instance:
uuid: "5e40c86f0adf4a18a8a1727c1759e229"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "8461d42cb9f4f737da8cf60612567420"
server_key_iv: "196bdd622afa4817c4becc51760b51e9"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.954926 15156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/wal/instance:
uuid: "5e40c86f0adf4a18a8a1727c1759e229"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "8461d42cb9f4f737da8cf60612567420"
server_key_iv: "196bdd622afa4817c4becc51760b51e9"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.958971 15156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:07:05.961529 15171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:05.962811 15156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:05.962968 15156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/wal
uuid: "5e40c86f0adf4a18a8a1727c1759e229"
format_stamp: "Formatted at 2026-08-07 05:07:05 on dist-test-slave-1x4g"
server_key: "8461d42cb9f4f737da8cf60612567420"
server_key_iv: "196bdd622afa4817c4becc51760b51e9"
server_key_version: "encryptionkey@0"
I20260807 05:07:05.963088 15156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:05.986385 15156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:05.990155 15156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:05.990388 15156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:05.998245 15156 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.252:41541
I20260807 05:07:05.998256 15225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.252:41541 every 8 connection(s)
I20260807 05:07:05.999310 15156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
I20260807 05:07:06.002120 15226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:06.002686 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15156
I20260807 05:07:06.002794 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786079068077062-1095-0/minicluster-data/master-2/wal/instance
I20260807 05:07:06.003116 1095 external_mini_cluster.cc:1428] Setting key ae4bfe0693dedd1df0a6dc2c387c5e0a
I20260807 05:07:06.006791 15226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:06.017897 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.008571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.1.17.254:43849)
0807 05:07:06.008705 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.008710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.009071 (+ 361us) server_negotiation.cc:407] Connection header received
0807 05:07:06.009236 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.009242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.009314 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.009412 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.010138 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.010646 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.011841 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.012231 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.012346 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.014574 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.014579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.014584 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.014613 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.016070 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.016516 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.016518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.016519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.016589 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.016936 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.016940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.016942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.017191 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.017295 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.017669 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.017767 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:07:06.018306 15071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } attempt: 3
I20260807 05:07:06.018561 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.004672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.1.17.252:41541)
0807 05:07:06.005459 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.005481 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.005531 (+ 50us) server_negotiation.cc:407] Connection header received
0807 05:07:06.006188 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.006206 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.006542 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.006895 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.007821 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.009153 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.010179 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.010724 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.010901 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.013586 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.013595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.013609 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.013632 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.015252 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.015691 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.015699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.015709 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.015807 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.016084 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.016088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.016090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.016617 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.016882 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.017185 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.017499 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":136,"threads_started":1}
I20260807 05:07:06.018565 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.008405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43849 (local address 127.0.0.1:51640)
0807 05:07:06.008924 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:06.008975 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:06.009113 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:06.009502 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:06.009507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:06.009837 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:06.009993 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.009999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.010774 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.010779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.011704 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.011730 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.012373 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.012376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.012453 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.012989 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:06.013004 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:06.014407 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:06.016162 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.016168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.016174 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.016418 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.016706 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.016712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.016715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.016829 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.017309 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:06.017315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:06.017419 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 05:07:06.017587 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 05:07:06.017744 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":170,"threads_started":1}
I20260807 05:07:06.018914 15147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } attempt: 3
I20260807 05:07:06.020891 15226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:06.028575 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.019323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57876)
0807 05:07:06.019477 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:06.019494 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:06.019582 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:06.020086 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:06.020089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:06.020271 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:06.020388 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.020395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.021408 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.021413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.022193 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.022208 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.022881 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.022885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.022972 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.023456 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:06.023470 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:06.024936 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:06.026880 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.026883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.026885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.027093 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.027400 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.027405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.027410 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.027520 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.028007 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:06.028012 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:06.028115 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 05:07:06.028204 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 05:07:06.028343 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
I20260807 05:07:06.029604 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.019364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57876 (local address 127.1.17.252:41541)
0807 05:07:06.019849 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.019853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.019867 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:07:06.019901 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.019903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.019950 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.020510 (+ 560us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.020610 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.021261 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.022360 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.022759 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.023183 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.025082 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.025086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.025088 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.025111 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.026770 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.027194 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.027198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.027200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.027289 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.027626 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.027633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.027637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.027890 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.029180 (+ 1290us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.029266 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.029392 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":424,"spinlock_wait_cycles":58112,"thread_start_us":114,"threads_started":1}
I20260807 05:07:06.030882 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.018735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57874)
0807 05:07:06.018951 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:06.018968 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:06.019043 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:06.020076 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:06.020079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:06.020245 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:06.020337 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.020343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.022693 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.022696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.023517 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.023530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.024151 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.024155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.024240 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.024684 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:06.024698 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:06.026239 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:06.029099 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.029103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.029105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.029359 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.029786 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.029791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.029794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.029891 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.030364 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:06.030367 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:06.030474 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 05:07:06.030566 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 05:07:06.030690 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260807 05:07:06.031767 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.018800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.1.17.252:41541)
0807 05:07:06.019836 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.019845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.019859 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:07:06.019890 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.019892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.019939 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.020470 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.020552 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.022561 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.023664 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.024032 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.024469 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.026391 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.026395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.026397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.026436 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.028954 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.029502 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.029507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.029509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.029599 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.030024 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.030029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.030031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.030247 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.031401 (+ 1154us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.031475 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.031583 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":970}
I20260807 05:07:06.032822 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.021727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.1.17.254:43849)
0807 05:07:06.021831 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.021835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.021848 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:07:06.021885 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.021888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.021927 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.022011 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.022755 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.023479 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.024642 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.025004 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.025076 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.028280 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.028283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.028286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.028312 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.030496 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.030974 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.030978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.030981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.031063 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.031391 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.031394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.031396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.031590 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.031784 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.032554 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.032663 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29,"spinlock_wait_cycles":768}
I20260807 05:07:06.033365 15147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: Bootstrap starting.
I20260807 05:07:06.034768 15071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: Bootstrap starting.
I20260807 05:07:06.035812 15147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:06.036542 15147 log.cc:824] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:06.036902 15071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:06.037568 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.021635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.1.17.253:41145)
0807 05:07:06.021856 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.021861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.022012 (+ 151us) server_negotiation.cc:407] Connection header received
0807 05:07:06.022168 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.022171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.022213 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.022259 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.022814 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.023862 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.025285 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.026324 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.026397 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.029297 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.029301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.029303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.029329 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.030942 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.031475 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.031479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.031482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.031566 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.031877 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.031882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.031885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.032134 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.032205 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.037301 (+ 5096us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.037417 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260807 05:07:06.037844 15071 log.cc:824] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:06.038020 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.021557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:41145 (local address 127.0.0.1:35288)
0807 05:07:06.021949 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:06.021965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:06.022068 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:06.022392 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:06.022394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:06.022552 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:06.022698 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.022705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.023999 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.024001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.025126 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.025142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.026543 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.026548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.026670 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.027667 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:06.027696 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:06.029141 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:06.031071 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.031076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.031079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.031349 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.031677 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.031680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.031682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.031763 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.032275 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:06.032277 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:06.032433 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 05:07:06.037234 (+ 4801us) client_negotiation.cc:240] Negotiation successful
0807 05:07:06.037370 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":24,"spinlock_wait_cycles":7552}
I20260807 05:07:06.038790 15226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:07:06.039734 15147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: No bootstrap required, opened a new log
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
I20260807 05:07:06.043540 15147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.043756 15147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:06.043812 15147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5b890976cf4ab989fd217c1faca793, State: Initialized, Role: FOLLOWER
I20260807 05:07:06.044378 15147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
Aug 07 05:07:06 dist-test-slave-1x4g krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079225, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
I20260807 05:07:06.045797 15147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:06.046250 15234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.046419 15234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:07:06.047740 15153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.037527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.1.17.253:41145)
0807 05:07:06.037662 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.037666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.037740 (+ 74us) server_negotiation.cc:407] Connection header received
0807 05:07:06.037910 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.037913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.037956 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.038034 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.039108 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.039606 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.040652 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.040993 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.041091 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.043872 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.043876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.043878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.043906 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.045693 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.046197 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.046203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.046205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.046302 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.046658 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.046661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.046663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.046902 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.047100 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.047444 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.047558 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:07:06.049600 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.039932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:41541 (local address 127.0.0.1:57888)
0807 05:07:06.040077 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:06.040092 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:06.040175 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:06.040506 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:06.040509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:06.040668 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:06.040750 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.040756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.042007 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.042013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.042845 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:06.042856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.043468 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.043475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:06.043575 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.044058 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:06.044073 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:06.045659 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:06.048113 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.048116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.048119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.048378 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.048696 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:06.048698 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:06.048700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:06.048795 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:06.049165 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:06.049168 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:06.049247 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 05:07:06.049323 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:07:06.049441 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 05:07:06.050051 15071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: No bootstrap required, opened a new log
I20260807 05:07:06.050356 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.040158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.1.17.252:41541)
0807 05:07:06.040295 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.040299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.040311 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:07:06.040343 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.040346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.040380 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.040465 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.040881 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.041871 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.043000 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.043322 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.043425 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.045811 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.045815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.045818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.045845 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.047749 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.048529 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.048532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.048533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.048609 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.048883 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.048886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.048887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.049075 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.049982 (+ 907us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.050060 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.050230 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 05:07:06.052829 15246 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:07:06.052928 15246 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:07:06.050000 15078 master_runner.cc:186] Error getting master registration for 127.1.17.254:43849: OK, instance_id { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" instance_seqno: 1786079225550816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:07:06.053259 15078 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:07:06.054296 15071 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.054580 15071 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:06.054649 15071 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8fd5c6895e24e17a52cfc30f3440a2d, State: Initialized, Role: FOLLOWER
I20260807 05:07:06.054996 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:07:06.055176 15071 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.056627 15071 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:06.056751 15226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229: Bootstrap starting.
I20260807 05:07:06.059208 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.059355 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:07:06.059405 15226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:06.060328 15226 log.cc:824] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229: Log is configured to *not* fsync() on all Append() calls
W20260807 05:07:06.062047 15263 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:07:06.062129 15263 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:07:06.062561 15226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229: No bootstrap required, opened a new log
I20260807 05:07:06.066111 15226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.066375 15226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:06.066473 15226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e40c86f0adf4a18a8a1727c1759e229, State: Initialized, Role: FOLLOWER
I20260807 05:07:06.066952 15226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.068272 15226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:06.064867 15001 master_runner.cc:186] Error getting master registration for 127.1.17.252:41541: OK, instance_id { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" instance_seqno: 1786079225995884 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:07:06.068912 15001 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:07:06.069360 15266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.069511 15266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 05:07:06.073570 15277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:07:06.073668 15277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:07:06.077219 15156 master_runner.cc:263] No leader master found from master 5e40c86f0adf4a18a8a1727c1759e229
I20260807 05:07:06.077287 15156 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:07:06.087025 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.069466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.1.17.254:43849)
0807 05:07:06.069599 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.069605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.077099 (+ 7494us) server_negotiation.cc:407] Connection header received
0807 05:07:06.077236 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.077239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.077289 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.077379 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.078391 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.079166 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.080313 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.080642 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.080711 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.082877 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.082881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.082884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.082924 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.084605 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.085149 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.085153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.085155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.085279 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.085633 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.085640 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.085643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.085896 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.086064 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.086453 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.086577 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:07:06.091329 15153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.069825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35306 (local address 127.1.17.253:41145)
0807 05:07:06.069934 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.069938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.077466 (+ 7528us) server_negotiation.cc:407] Connection header received
0807 05:07:06.077645 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.077650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.077700 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.077756 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.079016 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.079779 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.080880 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.081229 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.081309 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.086507 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.086510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.086513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.086540 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.088171 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.088708 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.088712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.088713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.088841 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.089143 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.089146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.089147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.089361 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.090653 (+ 1292us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.091017 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.091151 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 05:07:06.093441 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:06.071584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.1.17.252:41541)
0807 05:07:06.071684 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:06.071688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:06.078868 (+ 7180us) server_negotiation.cc:407] Connection header received
0807 05:07:06.078985 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:06.078988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:06.079032 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:06.079111 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:06.085411 (+ 6300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.086216 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.087619 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:06.087939 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:06.088030 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:06.089672 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:06.089676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:06.089679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:06.089703 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:06.091382 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.092083 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.092086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.092088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.092169 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:06.092436 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:06.092439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:06.092441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:06.092639 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:06.092743 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:06.093140 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 05:07:06.093257 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 05:07:06.137298 15266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:06.137542 15266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.138674 15266 leader_election.cc:288] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8fd5c6895e24e17a52cfc30f3440a2d (127.1.17.254:43849), 4b5b890976cf4ab989fd217c1faca793 (127.1.17.253:41145)
I20260807 05:07:06.139097 15045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e40c86f0adf4a18a8a1727c1759e229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" is_pre_election: true
I20260807 05:07:06.139254 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e40c86f0adf4a18a8a1727c1759e229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b5b890976cf4ab989fd217c1faca793" is_pre_election: true
I20260807 05:07:06.139441 15045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e40c86f0adf4a18a8a1727c1759e229 in term 0.
I20260807 05:07:06.139516 15121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e40c86f0adf4a18a8a1727c1759e229 in term 0.
I20260807 05:07:06.139881 15173 leader_election.cc:302] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e40c86f0adf4a18a8a1727c1759e229, a8fd5c6895e24e17a52cfc30f3440a2d; no voters:
I20260807 05:07:06.140381 15266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:07:06.140486 15266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:07:06.140534 15266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:06.141855 15266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.142174 15266 leader_election.cc:288] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [CANDIDATE]: Term 1 election: Requested vote from peers a8fd5c6895e24e17a52cfc30f3440a2d (127.1.17.254:43849), 4b5b890976cf4ab989fd217c1faca793 (127.1.17.253:41145)
I20260807 05:07:06.142561 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e40c86f0adf4a18a8a1727c1759e229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b5b890976cf4ab989fd217c1faca793"
I20260807 05:07:06.142679 15121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:06.143949 15121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e40c86f0adf4a18a8a1727c1759e229 in term 1.
I20260807 05:07:06.144287 15172 leader_election.cc:302] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b5b890976cf4ab989fd217c1faca793, 5e40c86f0adf4a18a8a1727c1759e229; no voters:
I20260807 05:07:06.144728 15045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e40c86f0adf4a18a8a1727c1759e229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d"
I20260807 05:07:06.144898 15045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:06.146227 15045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e40c86f0adf4a18a8a1727c1759e229 in term 1.
I20260807 05:07:06.148983 15266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:06.149272 15266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [term 1 LEADER]: Becoming Leader. State: Replica: 5e40c86f0adf4a18a8a1727c1759e229, State: Running, Role: LEADER
I20260807 05:07:06.149731 15266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } }
I20260807 05:07:06.151738 15266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e40c86f0adf4a18a8a1727c1759e229. Latest consensus state: current_term: 1 leader_uuid: "5e40c86f0adf4a18a8a1727c1759e229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.151862 15266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:06.152225 15285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:06.153172 15285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:06.162681 15121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [term 1 FOLLOWER]: Refusing update from remote peer 5e40c86f0adf4a18a8a1727c1759e229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:06.163355 15282 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:06.168641 15234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e40c86f0adf4a18a8a1727c1759e229. Latest consensus state: current_term: 1 leader_uuid: "5e40c86f0adf4a18a8a1727c1759e229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.168805 15234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:07:06.169927 15045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [term 1 FOLLOWER]: Refusing update from remote peer 5e40c86f0adf4a18a8a1727c1759e229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:06.170547 15266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:06.174467 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e40c86f0adf4a18a8a1727c1759e229. Latest consensus state: current_term: 1 leader_uuid: "5e40c86f0adf4a18a8a1727c1759e229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.174603 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:07:06.183075 15282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e40c86f0adf4a18a8a1727c1759e229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.183229 15282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:06.184441 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e40c86f0adf4a18a8a1727c1759e229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.184584 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:07:06.185014 15286 mvcc.cc:205] Tried to move back new op lower bound from 7315780510353543168 to 7315780510312681472. Current Snapshot: MvccSnapshot[applied={T|T < 7315780510353543168}]
I20260807 05:07:06.185406 15289 mvcc.cc:205] Tried to move back new op lower bound from 7315780510353543168 to 7315780510312681472. Current Snapshot: MvccSnapshot[applied={T|T < 7315780510353543168}]
I20260807 05:07:06.187347 15266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e40c86f0adf4a18a8a1727c1759e229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.187467 15266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:06.188608 15234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e40c86f0adf4a18a8a1727c1759e229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8fd5c6895e24e17a52cfc30f3440a2d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43849 } } peers { permanent_uuid: "4b5b890976cf4ab989fd217c1faca793" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 41145 } } peers { permanent_uuid: "5e40c86f0adf4a18a8a1727c1759e229" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 41541 } } }
I20260807 05:07:06.188750 15234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:07:06.191524 15287 mvcc.cc:205] Tried to move back new op lower bound from 7315780510353543168 to 7315780510312681472. Current Snapshot: MvccSnapshot[applied={T|T < 7315780510353543168}]
I20260807 05:07:06.195264 15285 catalog_manager.cc:1347] Generated new cluster ID: 60af7a71a8df4a80b7807800a78a7747
I20260807 05:07:06.195338 15285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:07:06.212478 15285 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:07:06.213796 15285 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:07:06.246425 15285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e40c86f0adf4a18a8a1727c1759e229: Generated new TSK 0
I20260807 05:07:06.247445 15285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:07:07.054266 15246 catalog_manager.cc:1259] Loaded cluster ID: 60af7a71a8df4a80b7807800a78a7747
I20260807 05:07:07.054355 15246 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: loading cluster ID for follower catalog manager: success
I20260807 05:07:07.056210 15246 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: acquiring CA information for follower catalog manager: success
I20260807 05:07:07.057548 15246 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4b5b890976cf4ab989fd217c1faca793: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:07:07.063161 15263 catalog_manager.cc:1259] Loaded cluster ID: 60af7a71a8df4a80b7807800a78a7747
I20260807 05:07:07.063217 15263 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: loading cluster ID for follower catalog manager: success
I20260807 05:07:07.064640 15263 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: acquiring CA information for follower catalog manager: success
I20260807 05:07:07.065677 15263 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a8fd5c6895e24e17a52cfc30f3440a2d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:07:07.318683 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15001
I20260807 05:07:07.327852 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15078
I20260807 05:07:07.335012 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15156
2026-08-07T05:07:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3133 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:07:07 dist-test-slave-1x4g krb5kdc[15303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:07:07 dist-test-slave-1x4g krb5kdc[15303](info): set up 2 sockets
Aug 07 05:07:07 dist-test-slave-1x4g krb5kdc[15303](info): commencing operation
krb5kdc: starting...
W20260807 05:07:08.381775 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:07:08 dist-test-slave-1x4g krb5kdc[15303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:07:08.504454 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:37185
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:45505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:37185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:08.615658 15316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:08.615903 15316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:08.615965 15316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:08.619905 15316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:08.619956 15316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:08.619974 15316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:08.619990 15316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:08.620047 15316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:08.624926 15316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:45505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:37185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:37185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:08.625952 15316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:08.626984 15316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:08.634992 15321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:08.635017 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:08.634944 15322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:08.635198 15316 server_base.cc:1034] running on GCE node
I20260807 05:07:08.635815 15316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:08.636957 15316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:08.638149 15316 hybrid_clock.cc:648] HybridClock initialized: now 1786079228638137 us; error 26 us; skew 500 ppm
Aug 07 05:07:08 dist-test-slave-1x4g krb5kdc[15303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079228, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:08.640928 15316 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:08.641767 15316 webserver.cc:460] Webserver started at http://127.1.17.254:42241/ using document root <none> and password file <none>
I20260807 05:07:08.642334 15316 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:08.642431 15316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:08.642635 15316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:08.644594 15316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "4f299761043743c6852708f2337d3444"
format_stamp: "Formatted at 2026-08-07 05:07:08 on dist-test-slave-1x4g"
server_key: "0a3581f2e4a38e8e15b22aaac83ae662"
server_key_iv: "b86a1332ea89e1a3cd69976746cabd3f"
server_key_version: "encryptionkey@0"
I20260807 05:07:08.645228 15316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "4f299761043743c6852708f2337d3444"
format_stamp: "Formatted at 2026-08-07 05:07:08 on dist-test-slave-1x4g"
server_key: "0a3581f2e4a38e8e15b22aaac83ae662"
server_key_iv: "b86a1332ea89e1a3cd69976746cabd3f"
server_key_version: "encryptionkey@0"
I20260807 05:07:08.648680 15316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:07:08.651088 15332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:08.652267 15316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:08.652400 15316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "4f299761043743c6852708f2337d3444"
format_stamp: "Formatted at 2026-08-07 05:07:08 on dist-test-slave-1x4g"
server_key: "0a3581f2e4a38e8e15b22aaac83ae662"
server_key_iv: "b86a1332ea89e1a3cd69976746cabd3f"
server_key_version: "encryptionkey@0"
I20260807 05:07:08.652534 15316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:08.693426 15316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:08.696899 15316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:08.697113 15316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:08.704532 15316 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:37185
I20260807 05:07:08.704561 15385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:37185 every 8 connection(s)
I20260807 05:07:08.705524 15316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:08.708237 15386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:08.710531 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15316
I20260807 05:07:08.710668 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:07:08.711000 1095 external_mini_cluster.cc:1428] Setting key 201fabd8ce89a4a43f980080e210cc48
I20260807 05:07:08.714378 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444: Bootstrap starting.
I20260807 05:07:08.716521 15386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:08.717314 15386 log.cc:824] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:08.719611 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444: No bootstrap required, opened a new log
Aug 07 05:07:08 dist-test-slave-1x4g krb5kdc[15303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:08.723630 15386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f299761043743c6852708f2337d3444" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37185 } }
I20260807 05:07:08.723917 15386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:08.723990 15386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f299761043743c6852708f2337d3444, State: Initialized, Role: FOLLOWER
I20260807 05:07:08.724407 15386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f299761043743c6852708f2337d3444" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37185 } }
I20260807 05:07:08.724527 15386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:07:08.724617 15386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:07:08.724735 15386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:08.725565 15386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f299761043743c6852708f2337d3444" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37185 } }
I20260807 05:07:08.725914 15386 leader_election.cc:302] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f299761043743c6852708f2337d3444; no voters:
I20260807 05:07:08.726240 15386 leader_election.cc:288] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:07:08.726308 15391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:08.726508 15391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [term 1 LEADER]: Becoming Leader. State: Replica: 4f299761043743c6852708f2337d3444, State: Running, Role: LEADER
I20260807 05:07:08.726701 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:08.712332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.1.17.254:37185)
0807 05:07:08.712730 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:08.712746 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:08.712822 (+ 76us) server_negotiation.cc:407] Connection header received
0807 05:07:08.713602 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:08.713623 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:08.713938 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:08.714297 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:08.715149 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:08.716174 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:08.717186 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:08.717747 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:08.717931 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:08.720610 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:08.720616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:08.720631 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:08.720660 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:08.723147 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:08.723554 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:08.723559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:08.723564 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:08.723704 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:08.723986 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:08.723992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:08.723994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:08.724476 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:08.725334 (+ 858us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:08.725470 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 05:07:08.725744 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":115,"threads_started":1}
I20260807 05:07:08.726785 15391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f299761043743c6852708f2337d3444" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37185 } }
I20260807 05:07:08.727451 15386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:08.729722 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:07:08.730080 15393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f299761043743c6852708f2337d3444. Latest consensus state: current_term: 1 leader_uuid: "4f299761043743c6852708f2337d3444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f299761043743c6852708f2337d3444" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37185 } } }
I20260807 05:07:08.730218 15393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:08.730888 15394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f299761043743c6852708f2337d3444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f299761043743c6852708f2337d3444" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37185 } } }
I20260807 05:07:08.731050 15394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444 [sys.catalog]: This master's current role is: LEADER
W20260807 05:07:08.732806 15407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:07:08.732880 15407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:07:08.732962 15408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:08.733783 15408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:08.738898 15408 catalog_manager.cc:1347] Generated new cluster ID: 7e176d9e6dbc4baaa0655ad040109e87
I20260807 05:07:08.738967 15408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:07:08.772526 15408 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:07:08.773748 15408 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:07:08.790868 15408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f299761043743c6852708f2337d3444: Generated new TSK 0
I20260807 05:07:08.791599 15408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:07:08.838519 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15316
2026-08-07T05:07:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1503 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:07:08 dist-test-slave-1x4g krb5kdc[15415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:07:08 dist-test-slave-1x4g krb5kdc[15415](info): set up 2 sockets
Aug 07 05:07:08 dist-test-slave-1x4g krb5kdc[15415](info): commencing operation
krb5kdc: starting...
W20260807 05:07:09.889249 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:07:09 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:07:10.013996 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38251
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:38251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:10.123556 15429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:10.123844 15429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:10.123960 15429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:10.127928 15429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:10.128011 15429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:10.128049 15429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:10.128106 15429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:10.128144 15429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:10.133136 15429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:38251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:10.134140 15429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:10.135174 15429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:10.143533 15437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:10.143595 15435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:10.143590 15434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:10.143755 15429 server_base.cc:1034] running on GCE node
I20260807 05:07:10.144450 15429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:10.145565 15429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:10.146727 15429 hybrid_clock.cc:648] HybridClock initialized: now 1786079230146699 us; error 39 us; skew 500 ppm
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:10.181984 15429 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:10.183053 15429 webserver.cc:460] Webserver started at http://127.1.17.254:37827/ using document root <none> and password file <none>
I20260807 05:07:10.183718 15429 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:10.183809 15429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:10.184059 15429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:10.186293 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "81531bc36a9441a7bea4004006683f2f"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "8d74a31495a103561cbd1301fcac730f"
server_key_iv: "de97814660f793a80116ab97cf3a923b"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.186874 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "81531bc36a9441a7bea4004006683f2f"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "8d74a31495a103561cbd1301fcac730f"
server_key_iv: "de97814660f793a80116ab97cf3a923b"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.190840 15429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:07:10.193440 15446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.194757 15429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:07:10.194908 15429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "81531bc36a9441a7bea4004006683f2f"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "8d74a31495a103561cbd1301fcac730f"
server_key_iv: "de97814660f793a80116ab97cf3a923b"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.195019 15429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:10.207772 15429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:10.210737 15429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:10.210893 15429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:10.217777 15429 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:38251
I20260807 05:07:10.218093 15500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:38251 every 8 connection(s)
I20260807 05:07:10.218772 15429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:10.220614 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15429
I20260807 05:07:10.220757 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:07:10.221084 1095 external_mini_cluster.cc:1428] Setting key a75e893ebf8b297c3697392bd6865925
I20260807 05:07:10.221830 15501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:10.228889 15501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Bootstrap starting.
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:10.231509 15501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:10.232450 15501 log.cc:824] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:10.234725 15501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: No bootstrap required, opened a new log
I20260807 05:07:10.236778 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.222427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.1.17.254:38251)
0807 05:07:10.222862 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.222881 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.222931 (+ 50us) server_negotiation.cc:407] Connection header received
0807 05:07:10.223765 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.223788 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.224200 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.224564 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:10.225374 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.226937 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.227823 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.228488 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.228639 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.231341 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:10.231349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:10.231369 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:10.231407 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:10.233463 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.233886 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.233891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.233896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.234037 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.234300 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.234303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.234305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.234838 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:10.235046 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.235278 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.235572 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":115,"threads_started":1}
I20260807 05:07:10.238658 15501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:10.238935 15501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:10.239041 15501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81531bc36a9441a7bea4004006683f2f, State: Initialized, Role: FOLLOWER
I20260807 05:07:10.239471 15501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:10.239638 15501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:07:10.239737 15501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:07:10.239832 15501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:10.240744 15501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:10.241153 15501 leader_election.cc:302] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81531bc36a9441a7bea4004006683f2f; no voters:
I20260807 05:07:10.241456 15501 leader_election.cc:288] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:07:10.241585 15506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:10.241834 15506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 LEADER]: Becoming Leader. State: Replica: 81531bc36a9441a7bea4004006683f2f, State: Running, Role: LEADER
I20260807 05:07:10.242204 15506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:10.242790 15501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:10.244091 15508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81531bc36a9441a7bea4004006683f2f. Latest consensus state: current_term: 1 leader_uuid: "81531bc36a9441a7bea4004006683f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } } }
I20260807 05:07:10.244102 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81531bc36a9441a7bea4004006683f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } } }
I20260807 05:07:10.244230 15508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:10.244234 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:10.244573 15515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:10.247810 15515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:10.253136 15515 catalog_manager.cc:1347] Generated new cluster ID: b4da8e36b3ea49ff983f5a5e91f01224
I20260807 05:07:10.253203 15515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:07:10.279942 15515 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:07:10.281296 15515 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:07:10.294247 15515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Generated new TSK 0
I20260807 05:07:10.294970 15515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:07:10.328915 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38251
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:10.444845 15527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:10.445106 15527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:10.445149 15527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:10.449141 15527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:10.449196 15527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:10.449304 15527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:07:10.454473 15527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:10.455566 15527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:10.456667 15527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:10.466298 15533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:10.466281 15535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:10.466281 15532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:10.466378 15527 server_base.cc:1034] running on GCE node
I20260807 05:07:10.466879 15527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:10.467545 15527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:10.468703 15527 hybrid_clock.cc:648] HybridClock initialized: now 1786079230468671 us; error 43 us; skew 500 ppm
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:10.471405 15527 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:07:10.472242 15527 webserver.cc:460] Webserver started at http://127.1.17.193:36575/ using document root <none> and password file <none>
I20260807 05:07:10.472735 15527 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:10.472855 15527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:10.473053 15527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:10.474960 15527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "92f769986b9d4eb494317a69114254b9"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "884db6a6fca394fd5108cc5660b6dfb4"
server_key_iv: "c2161a37a7731bb2d8a505ba97e36d62"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.475513 15527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "92f769986b9d4eb494317a69114254b9"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "884db6a6fca394fd5108cc5660b6dfb4"
server_key_iv: "c2161a37a7731bb2d8a505ba97e36d62"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.479163 15527 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 05:07:10.481528 15542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.482751 15527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:10.482892 15527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "92f769986b9d4eb494317a69114254b9"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "884db6a6fca394fd5108cc5660b6dfb4"
server_key_iv: "c2161a37a7731bb2d8a505ba97e36d62"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.482985 15527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:10.499065 15527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:10.502148 15527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:10.502317 15527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:10.502872 15527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:10.503728 15527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:10.503798 15527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.503891 15527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:10.503934 15527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.513769 15527 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:38959
I20260807 05:07:10.513796 15659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:38959 every 8 connection(s)
I20260807 05:07:10.514674 15527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:07:10.515985 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15527
I20260807 05:07:10.516121 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:07:10.516458 1095 external_mini_cluster.cc:1428] Setting key a2679c8cd689bed77b22e67c4a9cf59e
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:07:10.529649 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.516573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:39905 (local address 127.1.17.254:38251)
0807 05:07:10.516742 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.516747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.517703 (+ 956us) server_negotiation.cc:407] Connection header received
0807 05:07:10.519637 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.519639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.519682 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.519752 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:10.521242 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.521791 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.522977 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.523310 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.523432 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.525890 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:10.525894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:10.525897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:10.525924 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:10.527410 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.527983 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.527987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.527988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.528073 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.528428 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.528431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.528432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.528615 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:10.528724 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.529328 (+ 604us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.529475 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 05:07:10.530795 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.517039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:39905)
0807 05:07:10.517556 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:10.517599 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:10.519401 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:10.519943 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:10.519955 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:10.520490 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:10.521091 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.521106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.521940 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.521942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.522828 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.522838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.523474 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.523478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.523646 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.524247 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:10.524264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:10.525725 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:10.527549 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:10.527554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:10.527569 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:10.527872 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:10.528167 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:10.528171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:10.528173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:10.528316 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:10.528723 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:10.528729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:10.529078 (+ 349us) client_negotiation.cc:769] Sending connection context
0807 05:07:10.529266 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 05:07:10.529630 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":149,"threads_started":1}
I20260807 05:07:10.532316 15660 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38251
I20260807 05:07:10.532598 15660 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:10.533193 15660 heartbeater.cc:511] Master 127.1.17.254:38251 requested a full tablet report, sending...
I20260807 05:07:10.534849 15463 ts_manager.cc:194] Registered new tserver with Master: 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959)
I20260807 05:07:10.536626 15463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:39905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:07:10.547559 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38251
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:10.659317 15665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:10.659549 15665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:10.659615 15665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:10.663630 15665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:10.663679 15665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:10.663771 15665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:07:10.668860 15665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:10.669822 15665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:10.670814 15665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:07:10.680610 15665 server_base.cc:1034] running on GCE node
W20260807 05:07:10.680562 15670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:10.680562 15673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:10.680552 15671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:10.681249 15665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:10.681879 15665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:10.683072 15665 hybrid_clock.cc:648] HybridClock initialized: now 1786079230683055 us; error 33 us; skew 500 ppm
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:10.685809 15665 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:07:10.686705 15665 webserver.cc:460] Webserver started at http://127.1.17.194:42297/ using document root <none> and password file <none>
I20260807 05:07:10.687283 15665 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:10.687372 15665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:10.687604 15665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:10.689661 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "66818970d4f346ffae3527e3f600ee11"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "029196ff21a7dcb7a3273a27fdc7d9f0"
server_key_iv: "f29305973ca7f4d30af98477896f931f"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.690222 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "66818970d4f346ffae3527e3f600ee11"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "029196ff21a7dcb7a3273a27fdc7d9f0"
server_key_iv: "f29305973ca7f4d30af98477896f931f"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.693975 15665 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:07:10.696363 15681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.697633 15665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:10.697777 15665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "66818970d4f346ffae3527e3f600ee11"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "029196ff21a7dcb7a3273a27fdc7d9f0"
server_key_iv: "f29305973ca7f4d30af98477896f931f"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.697906 15665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:10.713202 15665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:10.716040 15665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:10.716240 15665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:10.716904 15665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:10.717748 15665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:10.717819 15665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.717886 15665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:10.717939 15665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.727028 15665 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:45299
I20260807 05:07:10.727069 15796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:45299 every 8 connection(s)
I20260807 05:07:10.727923 15665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:07:10.734046 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15665
I20260807 05:07:10.734187 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:07:10.734517 1095 external_mini_cluster.cc:1428] Setting key 28bbbcd50b8df69d890d100dd7edf3da
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:10.745574 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.731808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:43601 (local address 127.1.17.254:38251)
0807 05:07:10.731952 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.731957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.732173 (+ 216us) server_negotiation.cc:407] Connection header received
0807 05:07:10.733548 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.733552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.733613 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.733722 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:10.735332 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.736205 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.737921 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.738308 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.738432 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.741583 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:10.741586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:10.741588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:10.741610 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:10.743186 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.743849 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.743867 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.743872 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.743969 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.744393 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.744395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.744397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.744570 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:10.744646 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.745354 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.745458 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:07:10.746680 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.730711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:43601)
0807 05:07:10.732005 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:10.732058 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:10.733278 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:10.733937 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:10.733946 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:10.734463 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:10.735164 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.735186 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.736338 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.736345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.737802 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.737813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.738476 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.738482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.738710 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.739367 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:10.739392 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:10.741396 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:10.743298 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:10.743305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:10.743323 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:10.743714 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:10.744076 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:10.744081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:10.744085 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:10.744280 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:10.744682 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:10.744689 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:10.745074 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 05:07:10.745300 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 05:07:10.745663 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":798,"threads_started":1}
I20260807 05:07:10.747766 15797 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38251
I20260807 05:07:10.748049 15797 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:10.748665 15797 heartbeater.cc:511] Master 127.1.17.254:38251 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
I20260807 05:07:10.749774 15463 ts_manager.cc:194] Registered new tserver with Master: 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299)
I20260807 05:07:10.750370 15463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:43601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:07:10.764147 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38251
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:10.882201 15802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:10.882439 15802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:10.882486 15802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:10.886300 15802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:10.886345 15802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:10.886399 15802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:07:10.891325 15802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:10.892210 15802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:10.893149 15802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:10.902575 15807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:10.902511 15808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:10.902585 15802 server_base.cc:1034] running on GCE node
W20260807 05:07:10.902511 15812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:10.903296 15802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:10.903916 15802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:10.905108 15802 hybrid_clock.cc:648] HybridClock initialized: now 1786079230905049 us; error 42 us; skew 500 ppm
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:10.907891 15802 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:07:10.908723 15802 webserver.cc:460] Webserver started at http://127.1.17.195:43455/ using document root <none> and password file <none>
I20260807 05:07:10.909307 15802 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:10.909369 15802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:10.909593 15802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:10.911619 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "80766eaf2eae4ddcb0f19ab4549aa83b"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "2aa88f864be78aeb0cba828b038a0f46"
server_key_iv: "badd35584a4c5950162a47525f981f5f"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.912184 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "80766eaf2eae4ddcb0f19ab4549aa83b"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "2aa88f864be78aeb0cba828b038a0f46"
server_key_iv: "badd35584a4c5950162a47525f981f5f"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.915825 15802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:07:10.918246 15819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.919417 15802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:07:10.919535 15802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "80766eaf2eae4ddcb0f19ab4549aa83b"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "2aa88f864be78aeb0cba828b038a0f46"
server_key_iv: "badd35584a4c5950162a47525f981f5f"
server_key_version: "encryptionkey@0"
I20260807 05:07:10.919654 15802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:10.930336 15802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:10.933080 15802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:10.933265 15802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:10.933818 15802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:10.934628 15802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:10.934674 15802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.934741 15802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:10.934775 15802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:10.943867 15802 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:40179
I20260807 05:07:10.943893 15934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:40179 every 8 connection(s)
I20260807 05:07:10.944940 15802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:07:10.950505 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15802
I20260807 05:07:10.950654 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:07:10.950958 1095 external_mini_cluster.cc:1428] Setting key 0082a5ac61cda0c12690a8a129a0256c
Aug 07 05:07:10 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:10.958283 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.946699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:37947 (local address 127.1.17.254:38251)
0807 05:07:10.946850 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.946856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.947730 (+ 874us) server_negotiation.cc:407] Connection header received
0807 05:07:10.949021 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.949023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.949073 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.949148 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:10.950255 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.950928 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.952240 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.952532 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.952635 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.955055 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:10.955058 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:10.955060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:10.955083 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:10.956531 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.957058 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.957060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.957061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.957139 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.957404 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.957406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.957407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.957570 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:10.957639 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.958072 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.958167 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:07:10.959039 15937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.947140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:37947)
0807 05:07:10.947571 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:10.947612 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:10.948762 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:10.949308 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:10.949315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:10.949671 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:10.950129 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.950142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.951060 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.951064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.952111 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.952129 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.952685 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.952688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.952874 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.953496 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:10.953513 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:10.954908 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:10.956636 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:10.956640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:10.956651 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:10.956967 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:10.957216 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:10.957218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:10.957219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:10.957345 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:10.957653 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:10.957658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:10.957869 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 05:07:10.958009 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 05:07:10.958241 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
I20260807 05:07:10.960135 15935 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38251
I20260807 05:07:10.960343 15935 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:10.960914 15935 heartbeater.cc:511] Master 127.1.17.254:38251 requested a full tablet report, sending...
I20260807 05:07:10.961933 15463 ts_manager.cc:194] Registered new tserver with Master: 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179)
I20260807 05:07:10.962438 15463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:37947
I20260807 05:07:10.964974 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:07:10.974165 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.966927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.1.17.254:38251)
0807 05:07:10.967054 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.967057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.967259 (+ 202us) server_negotiation.cc:407] Connection header received
0807 05:07:10.967356 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.967358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.967392 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.967477 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:10.968148 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.968632 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.969498 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.969720 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.969843 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.971137 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:10.971139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:10.971141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:10.971162 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:10.972617 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.973112 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.973115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.973117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.973214 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:10.973448 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:10.973451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:10.973452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:10.973653 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:10.973752 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.973953 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.974054 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:07:10.976835 15463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:07:10.979911 15463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:07:10.994053 15947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.988135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:40179 (local address 127.0.0.1:41808)
0807 05:07:10.988453 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:10.988498 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:10.988651 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:10.989458 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:10.989465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:10.989491 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:10.989626 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.989634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.991022 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.991025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.992420 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.992431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.993541 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.993544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.993646 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.993680 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 05:07:10.993794 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 05:07:10.993913 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":31,"thread_start_us":95,"threads_started":1}
I20260807 05:07:10.994065 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.988287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.1.17.195:40179)
0807 05:07:10.988686 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.988700 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.988727 (+ 27us) server_negotiation.cc:407] Connection header received
0807 05:07:10.988836 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.988841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.989072 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.989237 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:10.989756 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.990881 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.992550 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.993424 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.993541 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.993690 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.993822 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.993917 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":201,"threads_started":1}
I20260807 05:07:10.994614 15949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.989243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38959 (local address 127.0.0.1:54914)
0807 05:07:10.989874 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:10.989887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:10.989969 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:10.990446 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:10.990449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:10.990467 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:10.990567 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.990576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.991962 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.991965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.992932 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.992940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.994219 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.994224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.994317 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.994340 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:07:10.994401 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 05:07:10.994479 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":83,"threads_started":1}
I20260807 05:07:10.995154 15951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.988963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:45299 (local address 127.0.0.1:39450)
0807 05:07:10.989781 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:10.989796 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:10.989962 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:10.990625 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:10.990630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:10.990648 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:10.990733 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.990741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.991831 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.991834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.993120 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:10.993133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.994767 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.994773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:10.994860 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.994882 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:07:10.994973 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 05:07:10.995040 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":48,"thread_start_us":82,"threads_started":1}
I20260807 05:07:10.995271 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.989175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.1.17.194:45299)
0807 05:07:10.989617 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.989630 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.989878 (+ 248us) server_negotiation.cc:407] Connection header received
0807 05:07:10.990175 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.990181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.990369 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.990523 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:10.990847 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.991672 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.993251 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.994030 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.994469 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.994605 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.995013 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.995112 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":138,"threads_started":1}
I20260807 05:07:10.995743 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:10.989658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54914 (local address 127.1.17.193:38959)
0807 05:07:10.990049 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:10.990068 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:10.990093 (+ 25us) server_negotiation.cc:407] Connection header received
0807 05:07:10.990181 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:10.990185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:10.990315 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:10.990467 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:10.990694 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.991804 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.993070 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:10.993877 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:10.993982 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:10.994127 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:10.994809 (+ 682us) server_negotiation.cc:299] Negotiation successful
0807 05:07:10.994898 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":90,"threads_started":1}
I20260807 05:07:10.997316 15868 tablet_service.cc:1467] Processing CreateTablet for tablet 49256ff61ca646aeb9f8926c6477c0e6 (DEFAULT_TABLE table=test-table [id=bee35f0aa5aa48e69c01df184e7d6195]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:10.998400 15868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49256ff61ca646aeb9f8926c6477c0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:10.998257 15731 tablet_service.cc:1467] Processing CreateTablet for tablet 49256ff61ca646aeb9f8926c6477c0e6 (DEFAULT_TABLE table=test-table [id=bee35f0aa5aa48e69c01df184e7d6195]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:10.998238 15593 tablet_service.cc:1467] Processing CreateTablet for tablet 49256ff61ca646aeb9f8926c6477c0e6 (DEFAULT_TABLE table=test-table [id=bee35f0aa5aa48e69c01df184e7d6195]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:10.999305 15593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49256ff61ca646aeb9f8926c6477c0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:10.999305 15731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49256ff61ca646aeb9f8926c6477c0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:11.004670 15953 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Bootstrap starting.
I20260807 05:07:11.006474 15954 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Bootstrap starting.
I20260807 05:07:11.007185 15953 tablet_bootstrap.cc:654] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:11.008584 15953 log.cc:824] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:11.009095 15955 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Bootstrap starting.
I20260807 05:07:11.009253 15954 tablet_bootstrap.cc:654] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:11.010109 15954 log.cc:824] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:11.010869 15953 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: No bootstrap required, opened a new log
I20260807 05:07:11.011041 15955 tablet_bootstrap.cc:654] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:11.011075 15953 ts_tablet_manager.cc:1397] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 05:07:11.012120 15954 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: No bootstrap required, opened a new log
I20260807 05:07:11.012101 15955 log.cc:824] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:11.012297 15954 ts_tablet_manager.cc:1397] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 05:07:11.014058 15955 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: No bootstrap required, opened a new log
I20260807 05:07:11.014258 15955 ts_tablet_manager.cc:1397] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:07:11.014976 15953 raft_consensus.cc:348] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.015260 15953 raft_consensus.cc:374] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:11.015319 15953 raft_consensus.cc:729] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80766eaf2eae4ddcb0f19ab4549aa83b, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.015785 15953 consensus_queue.cc:260] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.016081 15954 raft_consensus.cc:348] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.016337 15954 raft_consensus.cc:374] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:11.016400 15954 raft_consensus.cc:729] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66818970d4f346ffae3527e3f600ee11, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.016652 15935 heartbeater.cc:503] Master 127.1.17.254:38251 was elected leader, sending a full tablet report...
I20260807 05:07:11.016913 15953 ts_tablet_manager.cc:1428] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:07:11.016861 15954 consensus_queue.cc:260] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.017511 15797 heartbeater.cc:503] Master 127.1.17.254:38251 was elected leader, sending a full tablet report...
I20260807 05:07:11.017793 15954 ts_tablet_manager.cc:1428] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:07:11.018047 15955 raft_consensus.cc:348] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.018286 15955 raft_consensus.cc:374] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:11.018344 15955 raft_consensus.cc:729] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f769986b9d4eb494317a69114254b9, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.019029 15955 consensus_queue.cc:260] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.019589 15960 raft_consensus.cc:482] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:11.019644 15660 heartbeater.cc:503] Master 127.1.17.254:38251 was elected leader, sending a full tablet report...
I20260807 05:07:11.019721 15960 raft_consensus.cc:504] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.019968 15955 ts_tablet_manager.cc:1428] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:07:11.020901 15960 leader_election.cc:288] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179), 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959)
I20260807 05:07:11.025915 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.021461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:40179 (local address 127.1.17.194:50253)
0807 05:07:11.021603 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.021621 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.021702 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.022545 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.022548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.022565 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:11.022672 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.022680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.023643 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.023646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.024622 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.024631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.025501 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.025505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.025614 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.025640 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.025725 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.025797 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 05:07:11.025920 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.022092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:50253 (local address 127.1.17.195:40179)
0807 05:07:11.022244 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.022248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.022261 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:07:11.022305 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.022307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.022341 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.022431 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:11.022805 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.023536 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.024733 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.025372 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.025501 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.025555 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.025742 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.025818 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 05:07:11.026422 15888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49256ff61ca646aeb9f8926c6477c0e6" candidate_uuid: "66818970d4f346ffae3527e3f600ee11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" is_pre_election: true
I20260807 05:07:11.026670 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.021821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:50185 (local address 127.1.17.193:38959)
0807 05:07:11.021932 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.021936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.022517 (+ 581us) server_negotiation.cc:407] Connection header received
0807 05:07:11.022707 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.022710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.022750 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.022802 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:11.023329 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.023923 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.025162 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.025649 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.025756 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.025798 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.026506 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.026573 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 05:07:11.026684 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.021725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38959 (local address 127.1.17.194:50185)
0807 05:07:11.022422 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.022435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.022584 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.023046 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.023050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.023068 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:11.023226 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.023235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.024041 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.024044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.025036 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.025046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.026298 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.026301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.026375 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.026394 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.026488 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.026556 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":91,"threads_started":1}
I20260807 05:07:11.026675 15888 raft_consensus.cc:2454] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66818970d4f346ffae3527e3f600ee11 in term 0.
I20260807 05:07:11.027146 15682 leader_election.cc:302] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b; no voters:
I20260807 05:07:11.027398 15960 raft_consensus.cc:2781] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:07:11.027391 15613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49256ff61ca646aeb9f8926c6477c0e6" candidate_uuid: "66818970d4f346ffae3527e3f600ee11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f769986b9d4eb494317a69114254b9" is_pre_election: true
I20260807 05:07:11.027501 15960 raft_consensus.cc:482] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:07:11.027559 15960 raft_consensus.cc:3037] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:11.027660 15613 raft_consensus.cc:2454] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66818970d4f346ffae3527e3f600ee11 in term 0.
I20260807 05:07:11.028676 15960 raft_consensus.cc:504] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.029098 15960 leader_election.cc:288] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [CANDIDATE]: Term 1 election: Requested vote from peers 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179), 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959)
I20260807 05:07:11.029390 15888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49256ff61ca646aeb9f8926c6477c0e6" candidate_uuid: "66818970d4f346ffae3527e3f600ee11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b"
I20260807 05:07:11.029419 15613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49256ff61ca646aeb9f8926c6477c0e6" candidate_uuid: "66818970d4f346ffae3527e3f600ee11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f769986b9d4eb494317a69114254b9"
I20260807 05:07:11.029491 15888 raft_consensus.cc:3037] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:11.029558 15613 raft_consensus.cc:3037] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:11.030498 15888 raft_consensus.cc:2454] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66818970d4f346ffae3527e3f600ee11 in term 1.
I20260807 05:07:11.030514 15613 raft_consensus.cc:2454] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66818970d4f346ffae3527e3f600ee11 in term 1.
I20260807 05:07:11.030751 15682 leader_election.cc:302] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b; no voters:
I20260807 05:07:11.030915 15960 raft_consensus.cc:2781] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:11.031180 15960 raft_consensus.cc:686] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 1 LEADER]: Becoming Leader. State: Replica: 66818970d4f346ffae3527e3f600ee11, State: Running, Role: LEADER
I20260807 05:07:11.031486 15960 consensus_queue.cc:237] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.034412 15463 catalog_manager.cc:5495] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66818970d4f346ffae3527e3f600ee11 (127.1.17.194). New cstate: current_term: 1 leader_uuid: "66818970d4f346ffae3527e3f600ee11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } health_report { overall_health: UNKNOWN } } }
I20260807 05:07:11.051564 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.048388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.1.17.194:45299)
0807 05:07:11.048517 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.048521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.048533 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:07:11.048661 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.048663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.048712 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.048858 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:07:11.049061 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.049525 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.050294 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.050494 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.050599 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.050850 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:07:11.051237 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:07:11.051327 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.051399 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.051450 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:07:11.057222 15888 raft_consensus.cc:1261] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 FOLLOWER]: Refusing update from remote peer 66818970d4f346ffae3527e3f600ee11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:11.057222 15613 raft_consensus.cc:1261] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Refusing update from remote peer 66818970d4f346ffae3527e3f600ee11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:11.057937 15960 consensus_queue.cc:1035] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:11.058182 15964 consensus_queue.cc:1035] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:11.066601 15968 mvcc.cc:205] Tried to move back new op lower bound from 7315780530403078144 to 7315780530307571712. Current Snapshot: MvccSnapshot[applied={T|T < 7315780530403078144}]
I20260807 05:07:11.075112 15462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:11.075253 15462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:11.077652 15462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Sending DeleteTablet for 3 replicas of tablet 49256ff61ca646aeb9f8926c6477c0e6
I20260807 05:07:11.078472 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15527
I20260807 05:07:11.078408 15868 tablet_service.cc:1514] Processing DeleteTablet for tablet 49256ff61ca646aeb9f8926c6477c0e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:11 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:41808
I20260807 05:07:11.078696 15731 tablet_service.cc:1514] Processing DeleteTablet for tablet 49256ff61ca646aeb9f8926c6477c0e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:11 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:39450
I20260807 05:07:11.079874 15978 tablet_replica.cc:330] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: stopping tablet replica
I20260807 05:07:11.080157 15978 raft_consensus.cc:2229] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:07:11.080490 15978 raft_consensus.cc:2258] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:11.081717 15978 ts_tablet_manager.cc:1905] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:11.083793 15978 ts_tablet_manager.cc:1918] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:11.083920 15978 log.cc:1196] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/49256ff61ca646aeb9f8926c6477c0e6
I20260807 05:07:11.084203 15977 tablet_replica.cc:330] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: stopping tablet replica
I20260807 05:07:11.084311 15978 ts_tablet_manager.cc:1939] T 49256ff61ca646aeb9f8926c6477c0e6 P 66818970d4f346ffae3527e3f600ee11: Deleting consensus metadata
I20260807 05:07:11.084623 15977 raft_consensus.cc:2229] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:07:11.085026 15977 raft_consensus.cc:2258] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:11.085659 15447 catalog_manager.cc:4841] TS 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299): tablet 49256ff61ca646aeb9f8926c6477c0e6 (table test-table [id=bee35f0aa5aa48e69c01df184e7d6195]) successfully deleted
I20260807 05:07:11.086402 15977 ts_tablet_manager.cc:1905] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 05:07:11.088532 15447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:38959 (error 108)
I20260807 05:07:11.088637 15977 ts_tablet_manager.cc:1918] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:11.088732 15977 log.cc:1196] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/49256ff61ca646aeb9f8926c6477c0e6
I20260807 05:07:11.088828 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15665
I20260807 05:07:11.089128 15977 ts_tablet_manager.cc:1939] T 49256ff61ca646aeb9f8926c6477c0e6 P 80766eaf2eae4ddcb0f19ab4549aa83b: Deleting consensus metadata
I20260807 05:07:11.091456 15447 catalog_manager.cc:4841] TS 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179): tablet 49256ff61ca646aeb9f8926c6477c0e6 (table test-table [id=bee35f0aa5aa48e69c01df184e7d6195]) successfully deleted
I20260807 05:07:11.091687 15951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.091345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38959 (local address 127.0.0.1:54930)
0807 05:07:11.091495 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.091580 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 05:07:11.091850 15447 catalog_manager.cc:4584] TS 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49256ff61ca646aeb9f8926c6477c0e6: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38959: connect: Connection refused (error 111)
I20260807 05:07:11.096036 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15802
I20260807 05:07:11.102396 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15429
I20260807 05:07:11.110476 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38251
--webserver_interface=127.1.17.254
--webserver_port=37827
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:38251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:11.220279 15980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:11.220516 15980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:11.220554 15980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:11.224337 15980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:11.224385 15980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:11.224439 15980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:11.224493 15980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:11.224522 15980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:11.229470 15980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:38251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=37827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:11.230437 15980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:11.231424 15980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:11.246129 15987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:11.246134 15986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:11.246134 15989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:11.246374 15980 server_base.cc:1034] running on GCE node
I20260807 05:07:11.247011 15980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:11.248071 15980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:11.249258 15980 hybrid_clock.cc:648] HybridClock initialized: now 1786079231249213 us; error 55 us; skew 500 ppm
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:11.251957 15980 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:11.252869 15980 webserver.cc:460] Webserver started at http://127.1.17.254:37827/ using document root <none> and password file <none>
I20260807 05:07:11.253448 15980 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:11.253559 15980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:11.257529 15980 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:07:11.259903 15997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.261381 15980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 05:07:11.261552 15980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "81531bc36a9441a7bea4004006683f2f"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "8d74a31495a103561cbd1301fcac730f"
server_key_iv: "de97814660f793a80116ab97cf3a923b"
server_key_version: "encryptionkey@0"
I20260807 05:07:11.262228 15980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:11.280357 15980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:11.283473 15980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:11.283710 15980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:11.291893 15980 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:38251
I20260807 05:07:11.292147 16049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:38251 every 8 connection(s)
I20260807 05:07:11.293215 15980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:11.296970 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 15980
I20260807 05:07:11.297484 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:38959
--local_ip_for_outbound_sockets=127.1.17.193
--tserver_master_addrs=127.1.17.254:38251
--webserver_port=36575
--webserver_interface=127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:11.297994 16050 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:07:11.300238 16050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Bootstrap starting.
I20260807 05:07:11.318012 16050 log.cc:824] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:11.327471 16050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:07:11.327905 16050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Bootstrap complete.
I20260807 05:07:11.332453 16050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:11.332937 16050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81531bc36a9441a7bea4004006683f2f, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.333477 16050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:11.333593 16050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:07:11.333678 16050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:07:11.333803 16050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:07:11.335700 16050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:11.336052 16050 leader_election.cc:302] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81531bc36a9441a7bea4004006683f2f; no voters:
I20260807 05:07:11.336345 16050 leader_election.cc:288] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:07:11.336488 16055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:07:11.336808 16055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 LEADER]: Becoming Leader. State: Replica: 81531bc36a9441a7bea4004006683f2f, State: Running, Role: LEADER
I20260807 05:07:11.337157 16055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:11.337631 16050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:11.338788 16056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "81531bc36a9441a7bea4004006683f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } } }
I20260807 05:07:11.338792 16057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81531bc36a9441a7bea4004006683f2f. Latest consensus state: current_term: 2 leader_uuid: "81531bc36a9441a7bea4004006683f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } } }
I20260807 05:07:11.338920 16056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:11.338935 16057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:11.339455 16064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:11.344204 16064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:11.344905 16064 catalog_manager.cc:1259] Loaded cluster ID: b4da8e36b3ea49ff983f5a5e91f01224
I20260807 05:07:11.345142 16064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:07:11.346289 16064 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 05:07:11.349166 16064 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 05:07:11.353550 16064 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:07:11.354135 16064 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Loaded TSK: 0
I20260807 05:07:11.355087 16064 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:07:11.431121 16052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:11.431391 16052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:11.431479 16052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:11.435487 16052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:11.435565 16052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:11.435654 16052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:07:11.441299 16052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:38959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=36575
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:11.442384 16052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:11.443373 16052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:11.453081 16080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:11.453166 16079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:11.453101 16083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:11.453864 16052 server_base.cc:1034] running on GCE node
I20260807 05:07:11.454262 16052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:11.454989 16052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:11.456188 16052 hybrid_clock.cc:648] HybridClock initialized: now 1786079231456165 us; error 42 us; skew 500 ppm
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:11.459337 16052 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:07:11.460274 16052 webserver.cc:460] Webserver started at http://127.1.17.193:36575/ using document root <none> and password file <none>
I20260807 05:07:11.460909 16052 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:11.461023 16052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:11.465149 16052 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:07:11.467658 16090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.468995 16052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:07:11.469130 16052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "92f769986b9d4eb494317a69114254b9"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "884db6a6fca394fd5108cc5660b6dfb4"
server_key_iv: "c2161a37a7731bb2d8a505ba97e36d62"
server_key_version: "encryptionkey@0"
I20260807 05:07:11.469688 16052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:11.487578 16052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:11.491128 16052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:11.491328 16052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:11.491911 16052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:11.493095 16097 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:07:11.498582 16052 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:07:11.498672 16052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 05:07:11.498754 16052 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:07:11.500402 16052 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:07:11.500483 16052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 05:07:11.500754 16097 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Bootstrap starting.
I20260807 05:07:11.511188 16052 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:38959
I20260807 05:07:11.511322 16210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:38959 every 8 connection(s)
I20260807 05:07:11.512168 16052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:07:11.515573 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 16052
I20260807 05:07:11.516222 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:45299
--local_ip_for_outbound_sockets=127.1.17.194
--tserver_master_addrs=127.1.17.254:38251
--webserver_port=42297
--webserver_interface=127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:11.522605 16097 log.cc:824] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:11.530385 16097 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:07:11.530807 16097 tablet_bootstrap.cc:492] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Bootstrap complete.
I20260807 05:07:11.531555 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.514213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:41621)
0807 05:07:11.514699 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.514753 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.515560 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.516823 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.516832 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.517272 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:11.517736 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.517751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.519604 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.519608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.521029 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.521041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.521764 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.521768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.521983 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.522722 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:11.522750 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:11.524768 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:11.527822 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.527828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.527841 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.528185 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.528673 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.528676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.528678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.528871 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.529741 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:11.529747 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:11.530039 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.530231 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.530576 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":147,"threads_started":1}
I20260807 05:07:11.532308 16097 ts_tablet_manager.cc:1397] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.008s
I20260807 05:07:11.532493 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.514259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:41621 (local address 127.1.17.254:38251)
0807 05:07:11.514815 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.514834 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.514883 (+ 49us) server_negotiation.cc:407] Connection header received
0807 05:07:11.515996 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.516007 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.516290 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.516737 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:11.517912 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.519398 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.521156 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.521611 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.521730 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.525057 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:11.525067 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:11.525088 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:11.525132 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:11.527678 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.528340 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.528351 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.528359 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.528550 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.528999 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.529007 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.529010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.529584 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:11.529765 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.530523 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.530958 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":171,"threads_started":1}
I20260807 05:07:11.534902 16211 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38251
I20260807 05:07:11.535154 16211 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:11.535121 16097 raft_consensus.cc:348] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.535614 16097 raft_consensus.cc:729] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f769986b9d4eb494317a69114254b9, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.535703 16211 heartbeater.cc:511] Master 127.1.17.254:38251 requested a full tablet report, sending...
I20260807 05:07:11.536094 16097 consensus_queue.cc:260] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.537072 16097 ts_tablet_manager.cc:1428] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260807 05:07:11.538830 16014 ts_manager.cc:194] Registered new tserver with Master: 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959)
I20260807 05:07:11.542155 16014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:41621
I20260807 05:07:11.544608 16211 heartbeater.cc:503] Master 127.1.17.254:38251 was elected leader, sending a full tablet report...
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:07:11.552358 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.541648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38959 (local address 127.0.0.1:54936)
0807 05:07:11.542044 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.542087 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.542209 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.543099 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.543106 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.543563 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:11.543702 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.543712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.544977 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.544984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.545681 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.545690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.546381 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.546393 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.546511 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.547022 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:11.547038 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:11.548503 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:11.550620 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.550628 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.550632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.550848 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.551161 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.551163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.551164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.551239 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.551774 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:11.551783 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:11.551951 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.552101 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.552240 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":118,"threads_started":1}
I20260807 05:07:11.552562 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.542324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.1.17.193:38959)
0807 05:07:11.542611 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.542622 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.542644 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:07:11.542705 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.542710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.542910 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.543076 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:11.543843 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.544866 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.545808 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.546257 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.546378 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.548625 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:11.548636 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:11.548649 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:11.548676 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:11.550511 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.550958 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.550964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.550966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.551063 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.551327 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.551330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.551332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.551632 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:11.551779 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.552213 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.552396 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":114,"threads_started":1}
I20260807 05:07:11.553505 16141 tablet_service.cc:1514] Processing DeleteTablet for tablet 49256ff61ca646aeb9f8926c6477c0e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:11 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:54936
I20260807 05:07:11.554320 16220 tablet_replica.cc:330] stopping tablet replica
I20260807 05:07:11.554662 16220 raft_consensus.cc:2229] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:07:11.554927 16220 raft_consensus.cc:2258] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:11.556010 16220 ts_tablet_manager.cc:1905] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:11.560674 16220 ts_tablet_manager.cc:1918] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:11.560803 16220 log.cc:1196] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/49256ff61ca646aeb9f8926c6477c0e6
I20260807 05:07:11.561127 16220 ts_tablet_manager.cc:1939] T 49256ff61ca646aeb9f8926c6477c0e6 P 92f769986b9d4eb494317a69114254b9: Deleting consensus metadata
I20260807 05:07:11.563160 15998 catalog_manager.cc:4841] TS 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959): tablet 49256ff61ca646aeb9f8926c6477c0e6 (table test-table [id=bee35f0aa5aa48e69c01df184e7d6195]) successfully deleted
W20260807 05:07:11.643412 16215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:11.643625 16215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:11.643664 16215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:11.647622 16215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:11.647672 16215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:11.647766 16215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:07:11.652700 16215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:45299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=42297
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:11.653626 16215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:11.654624 16215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:11.663810 16228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:11.663823 16226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:11.663822 16225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:11.664029 16215 server_base.cc:1034] running on GCE node
I20260807 05:07:11.664451 16215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:11.665189 16215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:11.666378 16215 hybrid_clock.cc:648] HybridClock initialized: now 1786079231666350 us; error 42 us; skew 500 ppm
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:11.669253 16215 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:07:11.670112 16215 webserver.cc:460] Webserver started at http://127.1.17.194:42297/ using document root <none> and password file <none>
I20260807 05:07:11.670650 16215 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:11.670725 16215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:11.674822 16215 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:07:11.676895 16235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.678104 16215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:11.678246 16215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "66818970d4f346ffae3527e3f600ee11"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "029196ff21a7dcb7a3273a27fdc7d9f0"
server_key_iv: "f29305973ca7f4d30af98477896f931f"
server_key_version: "encryptionkey@0"
I20260807 05:07:11.678771 16215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:11.702320 16215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:11.705641 16215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:11.705853 16215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:11.706512 16215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:11.707365 16215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:11.707425 16215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.707497 16215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:11.707540 16215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.717243 16215 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:45299
I20260807 05:07:11.717271 16350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:45299 every 8 connection(s)
I20260807 05:07:11.718225 16215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:07:11.725589 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 16215
I20260807 05:07:11.726106 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:40179
--local_ip_for_outbound_sockets=127.1.17.195
--tserver_master_addrs=127.1.17.254:38251
--webserver_port=43455
--webserver_interface=127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:11.733075 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.720115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57803 (local address 127.1.17.254:38251)
0807 05:07:11.720298 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.720303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.721165 (+ 862us) server_negotiation.cc:407] Connection header received
0807 05:07:11.722492 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.722495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.722547 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.722621 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:11.723780 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.724313 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.725778 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.726203 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.726345 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.729017 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:11.729021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:11.729023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:11.729049 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:11.730807 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.731378 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.731381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.731383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.731468 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.731860 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.731863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.731865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.732077 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:11.732197 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.732703 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.732882 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 05:07:11.733872 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.720508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:57803)
0807 05:07:11.721003 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.721051 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.722249 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.722801 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.722807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.723163 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:11.723638 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.723652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.724438 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.724441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.725576 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.725591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.726403 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.726406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.726560 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.727160 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:11.727176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:11.728816 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:11.730941 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.730948 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.730962 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.731242 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.731581 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.731585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.731587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.731749 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.732196 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:11.732204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:11.732451 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.732641 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.732994 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":112,"threads_started":1}
I20260807 05:07:11.735219 16351 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38251
I20260807 05:07:11.735474 16351 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:11.735939 16351 heartbeater.cc:511] Master 127.1.17.254:38251 requested a full tablet report, sending...
I20260807 05:07:11.737048 16014 ts_manager.cc:194] Registered new tserver with Master: 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299)
I20260807 05:07:11.737982 16014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:57803
W20260807 05:07:11.838584 16354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:11.838836 16354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:11.838892 16354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:11.843016 16354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:11.843065 16354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:11.843119 16354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:07:11.848048 16354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:40179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=43455
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:11.849020 16354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:11.849962 16354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:11.859428 16360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:11.859567 16354 server_base.cc:1034] running on GCE node
W20260807 05:07:11.859433 16359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:11.859732 16362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:11.860193 16354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:11.860854 16354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:11.862027 16354 hybrid_clock.cc:648] HybridClock initialized: now 1786079231862013 us; error 29 us; skew 500 ppm
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:11.864717 16354 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:07:11.865610 16354 webserver.cc:460] Webserver started at http://127.1.17.195:43455/ using document root <none> and password file <none>
I20260807 05:07:11.866231 16354 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:11.866300 16354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:11.869865 16354 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:07:11.871795 16369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.873106 16354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:11.873215 16354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "80766eaf2eae4ddcb0f19ab4549aa83b"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "2aa88f864be78aeb0cba828b038a0f46"
server_key_iv: "badd35584a4c5950162a47525f981f5f"
server_key_version: "encryptionkey@0"
I20260807 05:07:11.873714 16354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:11.907186 16354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:11.910497 16354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:11.910699 16354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:11.911325 16354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:11.912140 16354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:11.912187 16354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.912252 16354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:11.912284 16354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:11.920801 16354 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:40179
I20260807 05:07:11.920898 16486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:40179 every 8 connection(s)
I20260807 05:07:11.921831 16354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:07:11.923767 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 16354
Aug 07 05:07:11 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:11.933394 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.924436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.1.17.254:38251)
0807 05:07:11.924691 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.924694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.924894 (+ 200us) server_negotiation.cc:407] Connection header received
0807 05:07:11.925060 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.925063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.925117 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.925225 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:11.926063 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.926903 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.927859 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.928098 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.928175 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.929651 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:11.929653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:11.929655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:11.929675 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:11.931494 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.931986 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.931988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.931989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.932065 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.932358 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.932362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.932364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.932594 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:11.932865 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.933039 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.933188 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
I20260807 05:07:11.936357 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.923624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:57173 (local address 127.1.17.254:38251)
0807 05:07:11.923788 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.923793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.925273 (+ 1480us) server_negotiation.cc:407] Connection header received
0807 05:07:11.926270 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.926272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.926310 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.926364 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:11.927717 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.928325 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.929591 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.929806 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.929878 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.932794 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:11.932799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:11.932802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:11.932828 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:11.934402 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.934957 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.934960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.934962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.935045 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:11.935406 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:11.935409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:11.935410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:11.935632 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:11.935709 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.936170 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.936254 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:07:11.937281 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.924029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:57173)
0807 05:07:11.925133 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.925193 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.926061 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.926561 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.926569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.927106 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:11.927575 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.927589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.928459 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.928462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.929469 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.929480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.929965 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.929968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.930154 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.930697 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:11.930724 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:11.932614 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:11.934539 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.934547 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.934563 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.934824 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.935140 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:11.935144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:11.935145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:11.935305 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:11.935730 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:11.935735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:11.935958 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.936124 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.936353 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"thread_start_us":708,"threads_started":1}
I20260807 05:07:11.938455 16487 heartbeater.cc:347] Connected to a master server at 127.1.17.254:38251
I20260807 05:07:11.938683 16487 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:11.939186 16487 heartbeater.cc:511] Master 127.1.17.254:38251 requested a full tablet report, sending...
I20260807 05:07:11.940158 16014 ts_manager.cc:194] Registered new tserver with Master: 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179)
I20260807 05:07:11.940690 16014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:57173
I20260807 05:07:11.946915 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:07:11.950570 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.947481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.1.17.254:38251)
0807 05:07:11.947588 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.947591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.947726 (+ 135us) server_negotiation.cc:407] Connection header received
0807 05:07:11.947843 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.947845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.947953 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.948031 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:07:11.948215 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.948661 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.949466 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.949658 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.949731 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.949991 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:07:11.950307 (+ 316us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:07:11.950390 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.950446 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.950486 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:07:11.951140 16014 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:07:11.954254 16014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:07:11.962256 16141 tablet_service.cc:1467] Processing CreateTablet for tablet a6bb161967f04ca782007e2709f53124 (DEFAULT_TABLE table=test-table [id=68b0b458a99f4ed69f017105f7010b62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:11.962962 16141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bb161967f04ca782007e2709f53124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:11.966048 16097 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Bootstrap starting.
I20260807 05:07:11.966974 16097 tablet_bootstrap.cc:654] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:11.967170 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.961356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:40179 (local address 127.0.0.1:41820)
0807 05:07:11.961824 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.961839 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.961938 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.962665 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.962668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.962689 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:11.962774 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.962779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.964274 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.964277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.965277 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.965287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.966652 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.966655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.966754 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.966799 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.966875 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.966960 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260807 05:07:11.967311 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.961026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:45299 (local address 127.0.0.1:39458)
0807 05:07:11.961447 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:11.961466 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:11.961552 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:11.962251 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:11.962255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:11.962274 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:11.962382 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.962389 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.964201 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.964205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.965394 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:11.965406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.966563 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.966570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:11.966971 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.966994 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:07:11.967071 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 05:07:11.967142 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":268}
I20260807 05:07:11.967751 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.961242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.1.17.194:45299)
0807 05:07:11.961738 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.961751 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.961775 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:07:11.961853 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.961859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.962075 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.962241 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:11.962542 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.964057 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.965543 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.966394 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.966722 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.966897 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.967468 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.967585 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":125,"threads_started":1}
I20260807 05:07:11.968200 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:11.961524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.1.17.195:40179)
0807 05:07:11.962158 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:11.962193 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:11.962219 (+ 26us) server_negotiation.cc:407] Connection header received
0807 05:07:11.962294 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:11.962299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:11.962508 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:11.962674 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:11.962905 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.964051 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.965413 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:11.966521 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:11.967724 (+ 1203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:11.967889 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:11.967958 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 05:07:11.968050 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":174,"threads_started":1}
I20260807 05:07:11.968745 16097 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: No bootstrap required, opened a new log
I20260807 05:07:11.968891 16097 ts_tablet_manager.cc:1397] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260807 05:07:11.969424 16097 raft_consensus.cc:348] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.969563 16097 raft_consensus.cc:374] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:11.969636 16097 raft_consensus.cc:729] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f769986b9d4eb494317a69114254b9, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.969792 16097 consensus_queue.cc:260] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.970141 16097 ts_tablet_manager.cc:1428] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 05:07:11.971042 16284 tablet_service.cc:1467] Processing CreateTablet for tablet a6bb161967f04ca782007e2709f53124 (DEFAULT_TABLE table=test-table [id=68b0b458a99f4ed69f017105f7010b62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:11.971284 16418 tablet_service.cc:1467] Processing CreateTablet for tablet a6bb161967f04ca782007e2709f53124 (DEFAULT_TABLE table=test-table [id=68b0b458a99f4ed69f017105f7010b62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:11.972244 16284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bb161967f04ca782007e2709f53124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:11.972338 16418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6bb161967f04ca782007e2709f53124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:11.979218 16500 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Bootstrap starting.
I20260807 05:07:11.979599 16501 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Bootstrap starting.
I20260807 05:07:11.981580 16501 tablet_bootstrap.cc:654] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:11.981946 16500 tablet_bootstrap.cc:654] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:11.982637 16500 log.cc:824] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:11.982692 16501 log.cc:824] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:11.984509 16500 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: No bootstrap required, opened a new log
I20260807 05:07:11.984730 16500 ts_tablet_manager.cc:1397] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:07:11.985903 16501 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: No bootstrap required, opened a new log
I20260807 05:07:11.986107 16501 ts_tablet_manager.cc:1397] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 05:07:11.988016 16500 raft_consensus.cc:348] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.988276 16500 raft_consensus.cc:374] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:11.988338 16500 raft_consensus.cc:729] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66818970d4f346ffae3527e3f600ee11, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.988837 16500 consensus_queue.cc:260] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.988873 16501 raft_consensus.cc:348] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.989123 16501 raft_consensus.cc:374] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:11.989199 16501 raft_consensus.cc:729] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80766eaf2eae4ddcb0f19ab4549aa83b, State: Initialized, Role: FOLLOWER
I20260807 05:07:11.989544 16351 heartbeater.cc:503] Master 127.1.17.254:38251 was elected leader, sending a full tablet report...
I20260807 05:07:11.989591 16501 consensus_queue.cc:260] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:11.989996 16500 ts_tablet_manager.cc:1428] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260807 05:07:11.990326 16487 heartbeater.cc:503] Master 127.1.17.254:38251 was elected leader, sending a full tablet report...
I20260807 05:07:11.990545 16501 ts_tablet_manager.cc:1428] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260807 05:07:12.013525 16212 tablet.cc:2369] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:07:12.083210 16505 raft_consensus.cc:482] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:12.083405 16505 raft_consensus.cc:504] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:12.084481 16505 leader_election.cc:288] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299), 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959)
I20260807 05:07:12.088600 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.084723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:45299 (local address 127.1.17.195:55145)
0807 05:07:12.084900 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.084917 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:12.085022 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:12.085454 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:12.085457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:12.085475 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:12.085598 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:12.085607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.086547 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.086551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:12.087463 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:12.087473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.088212 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.088215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:12.088315 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:12.088340 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 05:07:12.088407 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 05:07:12.088473 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":21}
I20260807 05:07:12.088622 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.084853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:55145 (local address 127.1.17.194:45299)
0807 05:07:12.085054 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:12.085059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:12.085077 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:07:12.085137 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:12.085140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:12.085200 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:12.085290 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:12.085728 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.086433 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.087584 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.088099 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.088185 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:12.088225 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:12.088435 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 05:07:12.088499 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 05:07:12.088894 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.085240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:50907 (local address 127.1.17.193:38959)
0807 05:07:12.085551 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:12.085557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:12.085577 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:07:12.085630 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:12.085636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:12.085715 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:12.085819 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:12.086135 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.086731 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.087825 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.088325 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.088451 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:12.088597 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:12.088638 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 05:07:12.088713 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
I20260807 05:07:12.089097 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.085108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38959 (local address 127.1.17.195:50907)
0807 05:07:12.085468 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.085480 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:12.085555 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:12.085853 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:12.085856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:12.085873 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:12.085996 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:12.086002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.086848 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.086851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:12.087704 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:12.087713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.088455 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.088458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:12.088544 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:12.088564 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 05:07:12.088623 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 05:07:12.088873 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":20,"thread_start_us":113,"threads_started":1}
I20260807 05:07:12.089345 16304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb161967f04ca782007e2709f53124" candidate_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66818970d4f346ffae3527e3f600ee11" is_pre_election: true
I20260807 05:07:12.089675 16304 raft_consensus.cc:2454] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80766eaf2eae4ddcb0f19ab4549aa83b in term 0.
I20260807 05:07:12.089655 16161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb161967f04ca782007e2709f53124" candidate_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f769986b9d4eb494317a69114254b9" is_pre_election: true
I20260807 05:07:12.089926 16161 raft_consensus.cc:2454] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80766eaf2eae4ddcb0f19ab4549aa83b in term 0.
I20260807 05:07:12.090178 16371 leader_election.cc:302] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b; no voters:
I20260807 05:07:12.090443 16505 raft_consensus.cc:2781] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:07:12.090521 16505 raft_consensus.cc:482] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:07:12.090550 16505 raft_consensus.cc:3037] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:12.091599 16505 raft_consensus.cc:504] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:12.091918 16505 leader_election.cc:288] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [CANDIDATE]: Term 1 election: Requested vote from peers 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299), 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959)
I20260807 05:07:12.092330 16161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb161967f04ca782007e2709f53124" candidate_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f769986b9d4eb494317a69114254b9"
I20260807 05:07:12.092335 16304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6bb161967f04ca782007e2709f53124" candidate_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66818970d4f346ffae3527e3f600ee11"
I20260807 05:07:12.092468 16161 raft_consensus.cc:3037] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:12.092500 16304 raft_consensus.cc:3037] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:12.093535 16161 raft_consensus.cc:2454] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80766eaf2eae4ddcb0f19ab4549aa83b in term 1.
I20260807 05:07:12.093603 16304 raft_consensus.cc:2454] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80766eaf2eae4ddcb0f19ab4549aa83b in term 1.
I20260807 05:07:12.093844 16371 leader_election.cc:302] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80766eaf2eae4ddcb0f19ab4549aa83b, 92f769986b9d4eb494317a69114254b9; no voters:
I20260807 05:07:12.094048 16505 raft_consensus.cc:2781] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:12.094264 16505 raft_consensus.cc:686] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 LEADER]: Becoming Leader. State: Replica: 80766eaf2eae4ddcb0f19ab4549aa83b, State: Running, Role: LEADER
I20260807 05:07:12.094609 16505 consensus_queue.cc:237] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:12.097857 16013 catalog_manager.cc:5495] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b reported cstate change: term changed from 0 to 1, leader changed from <none> to 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195). New cstate: current_term: 1 leader_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } health_report { overall_health: UNKNOWN } } }
I20260807 05:07:12.120045 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.116777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.1.17.195:40179)
0807 05:07:12.116905 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:12.116909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:12.116921 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:07:12.117065 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:12.117068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:12.117119 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:12.117206 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:07:12.117415 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.117913 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.118782 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:12.118982 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:12.119056 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:12.119296 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:07:12.119680 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:07:12.119763 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:12.119818 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 05:07:12.119862 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:07:12.125633 16161 raft_consensus.cc:1261] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Refusing update from remote peer 80766eaf2eae4ddcb0f19ab4549aa83b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:12.125636 16304 raft_consensus.cc:1261] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 1 FOLLOWER]: Refusing update from remote peer 80766eaf2eae4ddcb0f19ab4549aa83b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:12.126333 16505 consensus_queue.cc:1035] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:12.126598 16508 consensus_queue.cc:1035] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:12.131963 16511 mvcc.cc:205] Tried to move back new op lower bound from 7315780534779576320 to 7315780534662496256. Current Snapshot: MvccSnapshot[applied={T|T < 7315780534779576320}]
I20260807 05:07:12.132808 16512 mvcc.cc:205] Tried to move back new op lower bound from 7315780534779576320 to 7315780534662496256. Current Snapshot: MvccSnapshot[applied={T|T < 7315780534779576320}]
I20260807 05:07:12.135367 16513 mvcc.cc:205] Tried to move back new op lower bound from 7315780534779576320 to 7315780534662496256. Current Snapshot: MvccSnapshot[applied={T|T < 7315780534779576320}]
I20260807 05:07:12.140934 16013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:12.141088 16013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:12.143433 16013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Sending DeleteTablet for 3 replicas of tablet a6bb161967f04ca782007e2709f53124
I20260807 05:07:12.144112 16284 tablet_service.cc:1514] Processing DeleteTablet for tablet a6bb161967f04ca782007e2709f53124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:12 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:39458
I20260807 05:07:12.144186 16418 tablet_service.cc:1514] Processing DeleteTablet for tablet a6bb161967f04ca782007e2709f53124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:12 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:41820
I20260807 05:07:12.144315 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 16052
I20260807 05:07:12.149017 16521 tablet_replica.cc:330] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: stopping tablet replica
I20260807 05:07:12.149289 16523 tablet_replica.cc:330] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: stopping tablet replica
I20260807 05:07:12.149372 16521 raft_consensus.cc:2229] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:07:12.149672 16521 raft_consensus.cc:2258] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:12.149691 16523 raft_consensus.cc:2229] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:07:12.150137 16523 raft_consensus.cc:2258] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:12.151113 16521 ts_tablet_manager.cc:1905] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:12.151559 16523 ts_tablet_manager.cc:1905] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:12.153796 16521 ts_tablet_manager.cc:1918] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:12.154040 16521 log.cc:1196] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/a6bb161967f04ca782007e2709f53124
I20260807 05:07:12.154440 16521 ts_tablet_manager.cc:1939] T a6bb161967f04ca782007e2709f53124 P 66818970d4f346ffae3527e3f600ee11: Deleting consensus metadata
I20260807 05:07:12.154680 16523 ts_tablet_manager.cc:1918] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:12.154831 16523 log.cc:1196] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/a6bb161967f04ca782007e2709f53124
I20260807 05:07:12.155289 16523 ts_tablet_manager.cc:1939] T a6bb161967f04ca782007e2709f53124 P 80766eaf2eae4ddcb0f19ab4549aa83b: Deleting consensus metadata
I20260807 05:07:12.155751 15998 catalog_manager.cc:4841] TS 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299): tablet a6bb161967f04ca782007e2709f53124 (table test-table [id=68b0b458a99f4ed69f017105f7010b62]) successfully deleted
W20260807 05:07:12.156289 15998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:38959 (error 108)
I20260807 05:07:12.156817 15998 catalog_manager.cc:4841] TS 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179): tablet a6bb161967f04ca782007e2709f53124 (table test-table [id=68b0b458a99f4ed69f017105f7010b62]) successfully deleted
I20260807 05:07:12.157395 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.157087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38959 (local address 127.0.0.1:54952)
0807 05:07:12.157251 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.157315 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 05:07:12.157635 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 16215
W20260807 05:07:12.157570 15998 catalog_manager.cc:4584] TS 92f769986b9d4eb494317a69114254b9 (127.1.17.193:38959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6bb161967f04ca782007e2709f53124: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38959: connect: Connection refused (error 111)
I20260807 05:07:12.166795 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 16354
I20260807 05:07:12.173682 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 15980
I20260807 05:07:12.181090 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38251
--webserver_interface=127.1.17.254
--webserver_port=37827
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:38251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:12.290665 16525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:12.290892 16525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:12.290961 16525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:12.294934 16525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:12.294981 16525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:12.295001 16525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:12.295017 16525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:12.295069 16525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:12.300083 16525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:38251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=37827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:12.301105 16525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:12.302129 16525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:12.311057 16530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:12.311036 16531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:12.311038 16525 server_base.cc:1034] running on GCE node
W20260807 05:07:12.311028 16533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:12.311721 16525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:12.312857 16525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:12.314038 16525 hybrid_clock.cc:648] HybridClock initialized: now 1786079232314008 us; error 42 us; skew 500 ppm
Aug 07 05:07:12 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:12.317060 16525 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:12.317912 16525 webserver.cc:460] Webserver started at http://127.1.17.254:37827/ using document root <none> and password file <none>
I20260807 05:07:12.318482 16525 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:12.318543 16525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:12.322214 16525 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:12.324259 16540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:12.325512 16525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:12.325654 16525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "81531bc36a9441a7bea4004006683f2f"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "8d74a31495a103561cbd1301fcac730f"
server_key_iv: "de97814660f793a80116ab97cf3a923b"
server_key_version: "encryptionkey@0"
I20260807 05:07:12.326215 16525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:12.344856 16525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:12.348047 16525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:12.348244 16525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:12.355290 16525 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:38251
I20260807 05:07:12.355562 16593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:38251 every 8 connection(s)
I20260807 05:07:12.356395 16525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:12.357339 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 16525
I20260807 05:07:12.357864 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:38959
--local_ip_for_outbound_sockets=127.1.17.193
--tserver_master_addrs=127.1.17.254:38251
--webserver_port=36575
--webserver_interface=127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:12.361765 16594 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:07:12.363432 16594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Bootstrap starting.
I20260807 05:07:12.381755 16594 log.cc:824] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:12.393803 16594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:07:12.394191 16594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: Bootstrap complete.
I20260807 05:07:12.399901 16594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:12.400488 16594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81531bc36a9441a7bea4004006683f2f, State: Initialized, Role: FOLLOWER
I20260807 05:07:12.401079 16594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:12.401206 16594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:07:12.401292 16594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:07:12.401397 16594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:07:12.403465 16594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:12.403808 16594 leader_election.cc:302] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81531bc36a9441a7bea4004006683f2f; no voters:
I20260807 05:07:12.404161 16594 leader_election.cc:288] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 05:07:12.404248 16598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 3 FOLLOWER]: Leader election won for term 3
I20260807 05:07:12.404472 16598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [term 3 LEADER]: Becoming Leader. State: Replica: 81531bc36a9441a7bea4004006683f2f, State: Running, Role: LEADER
I20260807 05:07:12.404803 16598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } }
I20260807 05:07:12.405586 16594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:12.405835 16598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81531bc36a9441a7bea4004006683f2f. Latest consensus state: current_term: 3 leader_uuid: "81531bc36a9441a7bea4004006683f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } } }
I20260807 05:07:12.405953 16598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:12.406343 16600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "81531bc36a9441a7bea4004006683f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81531bc36a9441a7bea4004006683f2f" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38251 } } }
I20260807 05:07:12.406538 16600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:12.410655 16615 catalog_manager.cc:1259] Loaded cluster ID: b4da8e36b3ea49ff983f5a5e91f01224
I20260807 05:07:12.410737 16615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: loading cluster ID for follower catalog manager: success
W20260807 05:07:12.411700 16615 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 81531bc36a9441a7bea4004006683f2f: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260807 05:07:12.411791 16615 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260807 05:07:12.411985 16603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:12.414573 16603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:12.415081 16603 catalog_manager.cc:1259] Loaded cluster ID: b4da8e36b3ea49ff983f5a5e91f01224
I20260807 05:07:12.415160 16603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 05:07:12.415716 16603 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4ca3b449ad google::LogMessage::Fail() at ??:0
@ 0x7f4ca3b4788a google::LogMessage::SendToLog() at ??:0
@ 0x7f4ca3b444de google::LogMessage::Flush() at ??:0
@ 0x7f4ca3b45a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4cae483fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f4cae484d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4cae481579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f4cae4c5640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4ca5d963d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4ca5d96c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f4ca5d983f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4ca5d87512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4ca73cb6db start_thread at ??:0
@ 0x7f4ca264261f clone at ??:0
W20260807 05:07:12.505908 16596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:12.506237 16596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:12.506309 16596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:12.512343 16596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:12.512419 16596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:12.512511 16596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:07:12.520392 16596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:38959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=36575
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:12.521728 16596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:12.523029 16596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:12.533946 16638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:12.533979 16637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:12.533979 16640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:12.534452 16596 server_base.cc:1034] running on GCE node
I20260807 05:07:12.534907 16596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:12.535679 16596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:12.536878 16596 hybrid_clock.cc:648] HybridClock initialized: now 1786079232536851 us; error 32 us; skew 500 ppm
Aug 07 05:07:12 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:12.540267 16596 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:07:12.541281 16596 webserver.cc:460] Webserver started at http://127.1.17.193:36575/ using document root <none> and password file <none>
I20260807 05:07:12.541952 16596 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:12.542050 16596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:12.547170 16596 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:07:12.549865 16647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:12.551390 16596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:07:12.551504 16596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "92f769986b9d4eb494317a69114254b9"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "884db6a6fca394fd5108cc5660b6dfb4"
server_key_iv: "c2161a37a7731bb2d8a505ba97e36d62"
server_key_version: "encryptionkey@0"
I20260807 05:07:12.552182 16596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:12.585139 16596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:12.588543 16596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:12.588762 16596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:12.589459 16596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:12.590610 16654 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:07:12.596309 16596 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:07:12.596385 16596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 05:07:12.596460 16596 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:07:12.598954 16596 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:07:12.599035 16596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260807 05:07:12.599474 16654 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Bootstrap starting.
I20260807 05:07:12.611050 16596 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:38959
I20260807 05:07:12.611071 16766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:38959 every 8 connection(s)
I20260807 05:07:12.612265 16596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:07:12.616252 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 16596
I20260807 05:07:12.616770 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:45299
--local_ip_for_outbound_sockets=127.1.17.194
--tserver_master_addrs=127.1.17.254:38251
--webserver_port=42297
--webserver_interface=127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:12.623523 16654 log.cc:824] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:12.634080 16654 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:07:12.634567 16654 tablet_bootstrap.cc:492] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Bootstrap complete.
I20260807 05:07:12.636354 16654 ts_tablet_manager.cc:1397] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Time spent bootstrapping tablet: real 0.037s user 0.023s sys 0.010s
I20260807 05:07:12.640180 16654 raft_consensus.cc:348] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:12.640892 16654 raft_consensus.cc:729] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f769986b9d4eb494317a69114254b9, State: Initialized, Role: FOLLOWER
I20260807 05:07:12.641505 16654 consensus_queue.cc:260] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:12.642720 16654 ts_tablet_manager.cc:1428] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260807 05:07:12.662570 16769 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:07:12.614416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:45499)
0807 05:07:12.614954 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.615008 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:12.615820 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:12.661630 (+ 45810us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":162,"threads_started":1}
W20260807 05:07:12.663189 16767 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:38251 (0 consecutive failures): Network error: Failed to ping master at 127.1.17.254:38251: Client connection negotiation failed: client connection to 127.1.17.254:38251: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 05:07:12.663966 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.663683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:34827)
0807 05:07:12.663830 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.663901 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 05:07:12.664711 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.664489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:37417)
0807 05:07:12.664607 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.664654 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 05:07:12.665012 16767 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:07:12.753355 16770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:12.753612 16770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:12.753679 16770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:12.757741 16770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:12.757812 16770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:12.757910 16770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:07:12.763226 16770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:45299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=42297
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:12.764273 16770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:12.765400 16770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:12.774936 16778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:12.774951 16777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:12.774991 16780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:12.775194 16770 server_base.cc:1034] running on GCE node
I20260807 05:07:12.775763 16770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:12.776316 16770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:12.777489 16770 hybrid_clock.cc:648] HybridClock initialized: now 1786079232777468 us; error 32 us; skew 500 ppm
Aug 07 05:07:12 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:12.780181 16770 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:07:12.781176 16770 webserver.cc:460] Webserver started at http://127.1.17.194:42297/ using document root <none> and password file <none>
I20260807 05:07:12.781692 16770 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:12.781750 16770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:12.785601 16770 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:12.787565 16788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:12.788820 16770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:12.788944 16770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "66818970d4f346ffae3527e3f600ee11"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "029196ff21a7dcb7a3273a27fdc7d9f0"
server_key_iv: "f29305973ca7f4d30af98477896f931f"
server_key_version: "encryptionkey@0"
I20260807 05:07:12.789484 16770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:12.807356 16770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:12.810526 16770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:12.810736 16770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:12.811358 16770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:12.812193 16770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:12.812247 16770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:12.812319 16770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:12.812362 16770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:12.821899 16770 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:45299
I20260807 05:07:12.821934 16903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:45299 every 8 connection(s)
I20260807 05:07:12.822882 16770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:07:12.826781 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.824943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:52493)
0807 05:07:12.825409 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.825518 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":117,"threads_started":1}
I20260807 05:07:12.827036 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 16770
W20260807 05:07:12.827456 16904 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:38251 (0 consecutive failures): Network error: Failed to ping master at 127.1.17.254:38251: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
I20260807 05:07:12.827543 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:40179
--local_ip_for_outbound_sockets=127.1.17.195
--tserver_master_addrs=127.1.17.254:38251
--webserver_port=43455
--webserver_interface=127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:12.828259 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.828050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:41319)
0807 05:07:12.828154 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.828200 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 05:07:12.828971 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:12.828747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:38321)
0807 05:07:12.828865 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:12.828912 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 05:07:12.829144 16904 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:07:12.939038 16907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:12.939245 16907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:12.939282 16907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:12.943145 16907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:12.943195 16907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:12.943250 16907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:07:12.948318 16907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:40179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=43455
--tserver_master_addrs=127.1.17.254:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:12.949363 16907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:12.950362 16907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:12.959894 16913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:12.959904 16915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:12.959887 16912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:12.960237 16907 server_base.cc:1034] running on GCE node
I20260807 05:07:12.960638 16907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:12.961304 16907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:12.962493 16907 hybrid_clock.cc:648] HybridClock initialized: now 1786079232962479 us; error 25 us; skew 500 ppm
Aug 07 05:07:12 dist-test-slave-1x4g krb5kdc[15415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:12.965242 16907 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:07:12.966204 16907 webserver.cc:460] Webserver started at http://127.1.17.195:43455/ using document root <none> and password file <none>
I20260807 05:07:12.966800 16907 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:12.966893 16907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:12.970661 16907 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:07:12.972713 16922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:12.973999 16907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:12.974139 16907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "80766eaf2eae4ddcb0f19ab4549aa83b"
format_stamp: "Formatted at 2026-08-07 05:07:10 on dist-test-slave-1x4g"
server_key: "2aa88f864be78aeb0cba828b038a0f46"
server_key_iv: "badd35584a4c5950162a47525f981f5f"
server_key_version: "encryptionkey@0"
I20260807 05:07:12.974704 16907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:13.000473 16907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:13.003818 16907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:13.004057 16907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:13.004725 16907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:13.005666 16907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:13.005741 16907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:13.005811 16907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:13.005859 16907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:13.015081 16907 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:40179
I20260807 05:07:13.015106 17040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:40179 every 8 connection(s)
I20260807 05:07:13.016180 16907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:07:13.020188 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:13.018359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:53301)
0807 05:07:13.018851 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:13.018995 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":132,"threads_started":1}
W20260807 05:07:13.020642 17041 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:38251 (0 consecutive failures): Network error: Failed to ping master at 127.1.17.254:38251: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
I20260807 05:07:13.021279 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:13.021057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:39553)
0807 05:07:13.021169 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:13.021218 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 05:07:13.021848 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:13.021651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:47111)
0807 05:07:13.021749 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:13.021793 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 05:07:13.022012 17041 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 05:07:13.026448 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 16907
I20260807 05:07:13.666087 17045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:13.665615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:37781)
0807 05:07:13.665873 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:13.665927 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260807 05:07:13.830057 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:13.829668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:49143)
0807 05:07:13.829906 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:13.829961 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260807 05:07:14.022941 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:14.022511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:38049)
0807 05:07:14.022781 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:14.022837 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":66,"threads_started":1}
I20260807 05:07:14.189742 17048 raft_consensus.cc:482] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:14.189991 17048 raft_consensus.cc:504] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:14.191162 17048 leader_election.cc:288] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299), 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179)
Aug 07 05:07:14 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.194@KRBTEST.COM
Aug 07 05:07:14 dist-test-slave-1x4g krb5kdc[15415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.195@KRBTEST.COM
I20260807 05:07:14.208638 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:14.191663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:40179 (local address 127.1.17.193:34697)
0807 05:07:14.191936 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:14.191949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:14.192038 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:14.193944 (+ 1906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:14.193950 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:14.194577 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:14.195556 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:14.195580 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.197627 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.197632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:14.199370 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:14.199376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.200400 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.200405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:14.200526 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:14.201273 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:14.201286 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:14.203391 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:14.205782 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:14.205786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:14.205788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:14.206071 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:14.206510 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:14.206513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:14.206515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:14.206721 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:14.207476 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:14.207483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:14.207843 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 05:07:14.208067 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 05:07:14.208414 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":413,"spinlock_wait_cycles":29184,"thread_start_us":67,"threads_started":1}
I20260807 05:07:14.208642 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:14.191321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:45299 (local address 127.1.17.193:43501)
0807 05:07:14.191543 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:14.191555 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:14.191678 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:14.193944 (+ 2266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:14.193950 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:14.194577 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:14.195556 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:14.195580 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.197627 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.197632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:14.199361 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:14.199375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.200296 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.200302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:14.200508 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:14.201216 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:14.201232 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:14.203023 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:14.205651 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:14.205657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:14.205671 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:14.206081 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:14.206510 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:14.206513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:14.206515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:14.206721 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:14.207476 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:14.207482 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:14.207843 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 05:07:14.208067 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 05:07:14.208474 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":527,"spinlock_wait_cycles":21120,"thread_start_us":53,"threads_started":1}
I20260807 05:07:14.208695 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:14.191575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:43501 (local address 127.1.17.194:45299)
0807 05:07:14.191970 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:14.191992 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:14.192054 (+ 62us) server_negotiation.cc:407] Connection header received
0807 05:07:14.193012 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:14.193021 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:14.193486 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:14.193820 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:14.195761 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.197418 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.199533 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.200162 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.200294 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:14.203565 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:14.203570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:14.203581 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:14.203607 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:14.205524 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:14.206198 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:14.206204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:14.206210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:14.206387 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:14.206823 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:14.206826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:14.206828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:14.207345 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:14.207480 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:14.208211 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 05:07:14.208495 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":119,"threads_started":1}
I20260807 05:07:14.208695 17052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:14.191860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:34697 (local address 127.1.17.195:40179)
0807 05:07:14.192192 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:14.192214 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:14.192248 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:07:14.193074 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:14.193080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:14.193487 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:14.193820 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:14.195761 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.197418 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.199699 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:14.200257 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:14.200417 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:14.203565 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:14.203570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:14.203582 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:14.203613 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:14.205680 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:14.206198 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:14.206204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:14.206210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:14.206387 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:14.206823 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:14.206826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:14.206828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:14.207345 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:14.207480 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:14.208211 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 05:07:14.208495 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":127,"threads_started":1}
W20260807 05:07:14.212023 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
W20260807 05:07:14.212277 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
I20260807 05:07:14.212325 16648 leader_election.cc:302] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92f769986b9d4eb494317a69114254b9; no voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b
I20260807 05:07:14.212570 17048 raft_consensus.cc:2726] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:07:14.667193 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:14.666821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:51941)
0807 05:07:14.666992 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:14.667069 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 05:07:14.831314 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:14.830840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:56097)
0807 05:07:14.831139 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:14.831203 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260807 05:07:15.024185 17054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:15.023798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:45945)
0807 05:07:15.024035 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:15.024090 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":70,"threads_started":1}
I20260807 05:07:15.668466 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:15.667921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:60587)
0807 05:07:15.668310 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:15.668364 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":238,"threads_started":1}
I20260807 05:07:15.832458 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:15.832051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:56945)
0807 05:07:15.832302 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:15.832351 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":81,"threads_started":1}
I20260807 05:07:15.946780 17057 raft_consensus.cc:482] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:15.946907 17057 raft_consensus.cc:504] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:15.947310 17057 leader_election.cc:288] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299), 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179)
W20260807 05:07:15.948164 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
W20260807 05:07:15.948352 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
I20260807 05:07:15.948413 16648 leader_election.cc:302] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92f769986b9d4eb494317a69114254b9; no voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b
I20260807 05:07:15.948585 17057 raft_consensus.cc:2726] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:07:16.025342 17058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:16.024961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:38739)
0807 05:07:16.025204 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:16.025257 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260807 05:07:16.669873 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:16.669242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:43517)
0807 05:07:16.669640 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:16.669734 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":127,"threads_started":1}
I20260807 05:07:16.833698 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:16.833266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:38715)
0807 05:07:16.833556 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:16.833610 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
I20260807 05:07:17.026558 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:17.026020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:59217)
0807 05:07:17.026282 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:17.026337 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
I20260807 05:07:17.454934 17062 raft_consensus.cc:482] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:17.455063 17062 raft_consensus.cc:504] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:17.455477 17062 leader_election.cc:288] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299), 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179)
W20260807 05:07:17.456322 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
W20260807 05:07:17.456462 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
I20260807 05:07:17.456522 16648 leader_election.cc:302] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92f769986b9d4eb494317a69114254b9; no voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b
I20260807 05:07:17.456720 17062 raft_consensus.cc:2726] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:07:17.671226 17064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:17.670770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:40245)
0807 05:07:17.671078 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:17.671128 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":111,"threads_started":1}
I20260807 05:07:17.834761 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:17.834389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:58527)
0807 05:07:17.834624 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:17.834676 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260807 05:07:18.027799 17066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:18.027411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:33393)
0807 05:07:18.027663 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:18.027716 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
I20260807 05:07:18.672464 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:18.671999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:40275)
0807 05:07:18.672279 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:18.672351 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":93,"threads_started":1}
I20260807 05:07:18.835911 17068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:18.835470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:42851)
0807 05:07:18.835764 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:18.835817 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":118,"threads_started":1}
I20260807 05:07:19.028882 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:19.028442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:49747)
0807 05:07:19.028685 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:19.028733 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
I20260807 05:07:19.298744 17070 raft_consensus.cc:482] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:19.298877 17070 raft_consensus.cc:504] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:19.299366 17070 leader_election.cc:288] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299), 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179)
W20260807 05:07:19.300259 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
W20260807 05:07:19.300426 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
I20260807 05:07:19.300472 16648 leader_election.cc:302] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92f769986b9d4eb494317a69114254b9; no voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b
I20260807 05:07:19.300625 17070 raft_consensus.cc:2726] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:07:19.673664 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:19.673268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:50407)
0807 05:07:19.673515 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:19.673568 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260807 05:07:19.837255 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:19.836707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:57565)
0807 05:07:19.837109 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:19.837164 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":130,"threads_started":1}
I20260807 05:07:20.030193 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:20.029686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:37423)
0807 05:07:20.030045 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:20.030103 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":78,"threads_started":1}
I20260807 05:07:20.674829 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:20.674407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:60233)
0807 05:07:20.674666 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:20.674732 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
I20260807 05:07:20.838336 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:20.837953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:49673)
0807 05:07:20.838188 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:20.838243 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":76,"threads_started":1}
I20260807 05:07:21.031229 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:21.030855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:36573)
0807 05:07:21.031087 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:21.031139 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
I20260807 05:07:21.676061 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:21.675651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:51829)
0807 05:07:21.675902 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:21.675962 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260807 05:07:21.839600 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:21.839107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:40965)
0807 05:07:21.839450 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:21.839505 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":162,"threads_started":1}
I20260807 05:07:22.032765 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:22.032158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:52875)
0807 05:07:22.032532 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:22.032619 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":124,"threads_started":1}
I20260807 05:07:22.677441 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:22.677015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:54401)
0807 05:07:22.677288 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:22.677341 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":72,"threads_started":1}
I20260807 05:07:22.840718 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:22.840330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:57955)
0807 05:07:22.840572 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:22.840622 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260807 05:07:23.034229 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:23.033827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:35745)
0807 05:07:23.034093 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:23.034145 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":78,"threads_started":1}
I20260807 05:07:23.678658 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:23.678195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:45273)
0807 05:07:23.678491 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:23.678553 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":119,"threads_started":1}
I20260807 05:07:23.841856 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:23.841432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:42659)
0807 05:07:23.841706 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:23.841760 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
I20260807 05:07:24.035315 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:24.034912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:33293)
0807 05:07:24.035174 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:24.035225 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
I20260807 05:07:24.043985 17086 raft_consensus.cc:482] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:24.044099 17086 raft_consensus.cc:504] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66818970d4f346ffae3527e3f600ee11" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 45299 } } peers { permanent_uuid: "80766eaf2eae4ddcb0f19ab4549aa83b" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40179 } } peers { permanent_uuid: "92f769986b9d4eb494317a69114254b9" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38959 } }
I20260807 05:07:24.044523 17086 leader_election.cc:288] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299), 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179)
W20260807 05:07:24.045323 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80766eaf2eae4ddcb0f19ab4549aa83b (127.1.17.195:40179): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
W20260807 05:07:24.045442 16648 leader_election.cc:341] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66818970d4f346ffae3527e3f600ee11 (127.1.17.194:45299): Not found: Tablet not found: a6bb161967f04ca782007e2709f53124
I20260807 05:07:24.045481 16648 leader_election.cc:302] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92f769986b9d4eb494317a69114254b9; no voters: 66818970d4f346ffae3527e3f600ee11, 80766eaf2eae4ddcb0f19ab4549aa83b
I20260807 05:07:24.045624 17086 raft_consensus.cc:2726] T a6bb161967f04ca782007e2709f53124 P 92f769986b9d4eb494317a69114254b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:07:24.679869 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:24.679450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:41447)
0807 05:07:24.679708 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:24.679767 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260807 05:07:24.842952 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:24.842541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:38361)
0807 05:07:24.842808 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:24.842860 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":88,"threads_started":1}
I20260807 05:07:25.036701 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:25.036217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:46947)
0807 05:07:25.036541 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:25.036596 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":143,"threads_started":1}
I20260807 05:07:25.681180 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:25.680677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:51023)
0807 05:07:25.681023 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:25.681079 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":147,"threads_started":1}
I20260807 05:07:25.844106 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:25.843704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:42961)
0807 05:07:25.843959 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:25.844012 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260807 05:07:26.037806 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:26.037412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:45361)
0807 05:07:26.037648 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:26.037698 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260807 05:07:26.682344 17093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:26.681891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:41371)
0807 05:07:26.682157 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:26.682211 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260807 05:07:26.845219 17094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:26.844852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:48727)
0807 05:07:26.845081 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:26.845130 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260807 05:07:27.038918 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:27.038553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:52299)
0807 05:07:27.038781 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:27.038831 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":74,"threads_started":1}
I20260807 05:07:27.683714 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:27.683294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.193:55091)
0807 05:07:27.683560 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:27.683612 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
I20260807 05:07:27.846427 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:27.845997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.194:45641)
0807 05:07:27.846274 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:27.846326 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":112,"threads_started":1}
I20260807 05:07:28.031816 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 16596
I20260807 05:07:28.040246 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:28.039832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:38251 (local address 127.1.17.195:44489)
0807 05:07:28.040101 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:28.040155 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":92,"threads_started":1}
I20260807 05:07:28.041078 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 16770
I20260807 05:07:28.047979 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 16907
2026-08-07T05:07:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19210 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:07:28 dist-test-slave-1x4g krb5kdc[17101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:07:28 dist-test-slave-1x4g krb5kdc[17101](info): set up 2 sockets
Aug 07 05:07:28 dist-test-slave-1x4g krb5kdc[17101](info): commencing operation
krb5kdc: starting...
W20260807 05:07:29.101886 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:07:29 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:07:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:07:29.224453 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35931
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:29.332980 17115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:29.333247 17115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:29.333316 17115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:29.337284 17115 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 05:07:29.337350 17115 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 05:07:29.337376 17115 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 05:07:29.337405 17115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:29.337464 17115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:29.337512 17115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:29.337548 17115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:29.337585 17115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:29.342630 17115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:29.343658 17115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:29.344678 17115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:29.353653 17122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:29.353669 17121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:29.353729 17115 server_base.cc:1034] running on GCE node
W20260807 05:07:29.353669 17124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:29.354524 17115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:29.355499 17115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:29.356658 17115 hybrid_clock.cc:648] HybridClock initialized: now 1786079249356633 us; error 38 us; skew 500 ppm
Aug 07 05:07:29 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079249, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:29.359362 17115 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:29.360210 17115 webserver.cc:460] Webserver started at http://127.1.17.254:36225/ using document root <none> and password file <none>
I20260807 05:07:29.360742 17115 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:29.360874 17115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:29.361111 17115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:29.363026 17115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "c10bc2c1d41748deb8e794750be53354"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "099a91ad7ea7b3db29f1beb4b645565a"
server_key_iv: "efcad6dc00f59b3feb5f8a4b532808f7"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.363587 17115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "c10bc2c1d41748deb8e794750be53354"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "099a91ad7ea7b3db29f1beb4b645565a"
server_key_iv: "efcad6dc00f59b3feb5f8a4b532808f7"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.367213 17115 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:07:29.369522 17131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:29.370723 17115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:29.370851 17115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "c10bc2c1d41748deb8e794750be53354"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "099a91ad7ea7b3db29f1beb4b645565a"
server_key_iv: "efcad6dc00f59b3feb5f8a4b532808f7"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.370954 17115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:29.392554 17115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:29.395169 17115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:29.395386 17115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:29.402513 17115 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:35931
I20260807 05:07:29.402797 17183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:35931 every 8 connection(s)
I20260807 05:07:29.403469 17115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:29.406162 17184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:29.410485 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 17115
I20260807 05:07:29.410614 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:07:29.411005 1095 external_mini_cluster.cc:1428] Setting key 23b0bb87548d99f103db949e9c6f7c70
I20260807 05:07:29.412297 17184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Bootstrap starting.
I20260807 05:07:29.415014 17184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:29.415925 17184 log.cc:824] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:29.417989 17184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: No bootstrap required, opened a new log
Aug 07 05:07:29 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:29.422084 17184 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:29.422344 17184 raft_consensus.cc:374] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:29.422411 17184 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10bc2c1d41748deb8e794750be53354, State: Initialized, Role: FOLLOWER
I20260807 05:07:29.422899 17184 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:29.423043 17184 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:07:29.423127 17184 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:07:29.423226 17184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:29.424294 17184 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:29.424634 17184 leader_election.cc:302] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10bc2c1d41748deb8e794750be53354; no voters:
I20260807 05:07:29.425002 17184 leader_election.cc:288] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:07:29.425064 17190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:29.425257 17190 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 LEADER]: Becoming Leader. State: Replica: c10bc2c1d41748deb8e794750be53354, State: Running, Role: LEADER
I20260807 05:07:29.425545 17190 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:29.425985 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:29.412481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.1.17.254:35931)
0807 05:07:29.412919 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:29.412938 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:29.412978 (+ 40us) server_negotiation.cc:407] Connection header received
0807 05:07:29.413841 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:29.413863 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:29.414248 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:29.414592 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:29.415320 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.416694 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.417612 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.418293 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.418471 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:29.420824 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:29.420831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:29.420847 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:29.420877 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:29.422767 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:29.423256 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:29.423262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:29.423267 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:29.423422 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:29.423690 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:29.423693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:29.423694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:29.424190 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:29.424372 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:29.424657 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 05:07:29.425001 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":110,"threads_started":1}
I20260807 05:07:29.426266 17184 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:29.427150 17191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10bc2c1d41748deb8e794750be53354. Latest consensus state: current_term: 1 leader_uuid: "c10bc2c1d41748deb8e794750be53354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } } }
I20260807 05:07:29.427515 17192 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c10bc2c1d41748deb8e794750be53354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } } }
I20260807 05:07:29.427640 17192 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:29.427712 17191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:29.428321 17199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:29.431120 17199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:29.436437 17199 catalog_manager.cc:1347] Generated new cluster ID: 9e1c73cd8da244d2b960ccde28abe0dd
I20260807 05:07:29.436496 17199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:07:29.445917 17199 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:07:29.447068 17199 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:07:29.472164 17199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Generated new TSK 0
I20260807 05:07:29.472942 17199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:07:29.509842 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35931
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:29.639411 17212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:29.639672 17212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:29.639746 17212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:29.643572 17212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:29.643663 17212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:29.643759 17212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:07:29.648976 17212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:29.649937 17212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:29.650902 17212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:29.660475 17219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:29.660540 17221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:29.660533 17212 server_base.cc:1034] running on GCE node
W20260807 05:07:29.660486 17218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:29.661118 17212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:29.661731 17212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:29.662938 17212 hybrid_clock.cc:648] HybridClock initialized: now 1786079249662922 us; error 34 us; skew 500 ppm
Aug 07 05:07:29 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079249, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:29.665772 17212 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:07:29.666605 17212 webserver.cc:460] Webserver started at http://127.1.17.193:42551/ using document root <none> and password file <none>
I20260807 05:07:29.667179 17212 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:29.667270 17212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:29.667511 17212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:29.669488 17212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "68369020f360442789263055a3e2d552"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "dc23ed44d48b1257987c9dbcd2b44814"
server_key_iv: "fc53c65160376f967b16e6a3a340f305"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.670032 17212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "68369020f360442789263055a3e2d552"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "dc23ed44d48b1257987c9dbcd2b44814"
server_key_iv: "fc53c65160376f967b16e6a3a340f305"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.673566 17212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:07:29.675941 17228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:29.677320 17212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:29.677455 17212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "68369020f360442789263055a3e2d552"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "dc23ed44d48b1257987c9dbcd2b44814"
server_key_iv: "fc53c65160376f967b16e6a3a340f305"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.677564 17212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:29.698729 17212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:29.701577 17212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:29.701790 17212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:29.702404 17212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:29.703238 17212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:29.703308 17212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:29.703378 17212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:29.703420 17212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:29.712440 17212 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:37695
I20260807 05:07:29.712498 17344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:37695 every 8 connection(s)
I20260807 05:07:29.713371 17212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:07:29.717595 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 17212
I20260807 05:07:29.717818 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:07:29.718164 1095 external_mini_cluster.cc:1428] Setting key f609c76efea1387db256b796f89e623e
Aug 07 05:07:29 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079249, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:29.728173 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:29.715075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:35845 (local address 127.1.17.254:35931)
0807 05:07:29.715210 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:29.715214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:29.716007 (+ 793us) server_negotiation.cc:407] Connection header received
0807 05:07:29.717435 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:29.717439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:29.717508 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:29.717620 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:29.719203 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.719822 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.721086 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.721330 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.721430 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:29.724001 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:29.724004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:29.724006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:29.724029 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:29.725604 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:29.726411 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:29.726414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:29.726416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:29.726497 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:29.726910 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:29.726912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:29.726913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:29.727087 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:29.727226 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:29.727876 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 05:07:29.728005 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:07:29.729187 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:29.715413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:35845)
0807 05:07:29.715850 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:29.715891 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:29.717189 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:29.717819 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:29.717834 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:29.718381 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:29.719060 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:29.719076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.719972 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.719975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:29.720957 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:29.720967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.721483 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.721487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:29.721667 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:29.722243 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:29.722259 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:29.723728 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:29.725735 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:29.725745 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:29.725765 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:29.726212 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:29.726599 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:29.726602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:29.726605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:29.726803 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:29.727266 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:29.727271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:29.727612 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 05:07:29.727833 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 05:07:29.728192 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":116,"threads_started":1}
I20260807 05:07:29.730579 17345 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35931
I20260807 05:07:29.730806 17345 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:29.731262 17345 heartbeater.cc:511] Master 127.1.17.254:35931 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:07:29.732920 17148 ts_manager.cc:194] Registered new tserver with Master: 68369020f360442789263055a3e2d552 (127.1.17.193:37695)
I20260807 05:07:29.734119 17148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:35845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:07:29.747954 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35931
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:29.860296 17351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:29.860553 17351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:29.860611 17351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:29.864660 17351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:29.864711 17351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:29.864809 17351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:07:29.869904 17351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:29.870846 17351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:29.871835 17351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:29.881510 17357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:29.881678 17351 server_base.cc:1034] running on GCE node
W20260807 05:07:29.881563 17356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:29.881510 17360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:29.882272 17351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:29.882933 17351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:29.884130 17351 hybrid_clock.cc:648] HybridClock initialized: now 1786079249884116 us; error 23 us; skew 500 ppm
Aug 07 05:07:29 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079249, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:29.886936 17351 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:07:29.887754 17351 webserver.cc:460] Webserver started at http://127.1.17.194:38861/ using document root <none> and password file <none>
I20260807 05:07:29.888309 17351 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:29.888415 17351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:29.888645 17351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:29.890682 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "8be603960c4340359a30369bf3e1c8eb"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "39a5f06c43293e428489007811887e0b"
server_key_iv: "f786ea9331f0f510ac916d019f18b73b"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.891242 17351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "8be603960c4340359a30369bf3e1c8eb"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "39a5f06c43293e428489007811887e0b"
server_key_iv: "f786ea9331f0f510ac916d019f18b73b"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.894965 17351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:07:29.897392 17367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:29.898598 17351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:07:29.898693 17351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "8be603960c4340359a30369bf3e1c8eb"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "39a5f06c43293e428489007811887e0b"
server_key_iv: "f786ea9331f0f510ac916d019f18b73b"
server_key_version: "encryptionkey@0"
I20260807 05:07:29.898774 17351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:29.931422 17351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:29.934264 17351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:29.934417 17351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:29.935029 17351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:29.935840 17351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:29.935889 17351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:29.935926 17351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:29.935952 17351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:29.944628 17351 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:42299
I20260807 05:07:29.944655 17482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:42299 every 8 connection(s)
I20260807 05:07:29.945549 17351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
Aug 07 05:07:29 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079249, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:29.955145 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 17351
I20260807 05:07:29.955261 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:07:29.955591 1095 external_mini_cluster.cc:1428] Setting key 138fda4669031468aea32a523ba25421
I20260807 05:07:29.959586 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:29.947299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:48095 (local address 127.1.17.254:35931)
0807 05:07:29.947480 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:29.947485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:29.948760 (+ 1275us) server_negotiation.cc:407] Connection header received
0807 05:07:29.949671 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:29.949674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:29.949720 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:29.949791 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:29.951051 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.951578 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.952647 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.952925 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.953001 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:29.955490 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:29.955493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:29.955495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:29.955515 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:29.957085 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:29.957693 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:29.957695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:29.957697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:29.957771 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:29.958196 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:29.958199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:29.958200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:29.958408 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:29.958487 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:29.959182 (+ 695us) server_negotiation.cc:299] Negotiation successful
0807 05:07:29.959353 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:07:29.960436 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:29.947746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:48095)
0807 05:07:29.948608 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:29.948652 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:29.949485 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:29.949967 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:29.949973 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:29.950426 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:29.950923 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:29.950936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.951695 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.951697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:29.952547 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:29.952556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:29.953050 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:29.953052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:29.953202 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:29.953804 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:29.953820 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:29.955331 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:29.957197 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:29.957203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:29.957219 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:29.957587 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:29.957870 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:29.957875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:29.957880 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:29.958100 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:29.958509 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:29.958519 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:29.958849 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 05:07:29.959099 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 05:07:29.959480 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":534,"threads_started":1}
I20260807 05:07:29.961692 17483 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35931
I20260807 05:07:29.961969 17483 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:29.962648 17483 heartbeater.cc:511] Master 127.1.17.254:35931 requested a full tablet report, sending...
I20260807 05:07:29.963845 17148 ts_manager.cc:194] Registered new tserver with Master: 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299)
I20260807 05:07:29.964694 17148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:48095
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:07:29.986146 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35931
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:30.096803 17488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:30.097051 17488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:30.097092 17488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:30.100909 17488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:30.100956 17488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:30.101011 17488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:07:30.106113 17488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:30.107061 17488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:30.108011 17488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:30.117416 17494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:30.117461 17496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:30.117422 17493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:30.117686 17488 server_base.cc:1034] running on GCE node
I20260807 05:07:30.118240 17488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:30.118767 17488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:30.119912 17488 hybrid_clock.cc:648] HybridClock initialized: now 1786079250119895 us; error 28 us; skew 500 ppm
Aug 07 05:07:30 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079250, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:30.122700 17488 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:07:30.123497 17488 webserver.cc:460] Webserver started at http://127.1.17.195:44717/ using document root <none> and password file <none>
I20260807 05:07:30.124058 17488 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:30.124117 17488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:30.124336 17488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:07:30.126528 17488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "06a14d884ecc4421a15e3f8a3120becc"
format_stamp: "Formatted at 2026-08-07 05:07:30 on dist-test-slave-1x4g"
server_key: "caf4ffd83a5075d78eb1c9a3f86db1f1"
server_key_iv: "3b7d3f4b2964a89d4d0188862ead3186"
server_key_version: "encryptionkey@0"
I20260807 05:07:30.127161 17488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "06a14d884ecc4421a15e3f8a3120becc"
format_stamp: "Formatted at 2026-08-07 05:07:30 on dist-test-slave-1x4g"
server_key: "caf4ffd83a5075d78eb1c9a3f86db1f1"
server_key_iv: "3b7d3f4b2964a89d4d0188862ead3186"
server_key_version: "encryptionkey@0"
I20260807 05:07:30.131098 17488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 05:07:30.133529 17503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:30.134675 17488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:30.134810 17488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "06a14d884ecc4421a15e3f8a3120becc"
format_stamp: "Formatted at 2026-08-07 05:07:30 on dist-test-slave-1x4g"
server_key: "caf4ffd83a5075d78eb1c9a3f86db1f1"
server_key_iv: "3b7d3f4b2964a89d4d0188862ead3186"
server_key_version: "encryptionkey@0"
I20260807 05:07:30.134943 17488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:30.164381 17488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:30.167307 17488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:30.167516 17488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:30.168123 17488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:30.169085 17488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:30.169147 17488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:30.169202 17488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:30.169225 17488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:30.178043 17488 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:33751
I20260807 05:07:30.178102 17621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:33751 every 8 connection(s)
I20260807 05:07:30.178895 17488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:07:30.183216 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 17488
I20260807 05:07:30.183350 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:07:30.183693 1095 external_mini_cluster.cc:1428] Setting key e0ded5f2107a5ffda49be389d2479bdb
Aug 07 05:07:30 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079250, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:30.193128 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.180829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:40541 (local address 127.1.17.254:35931)
0807 05:07:30.180986 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.180990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.181881 (+ 891us) server_negotiation.cc:407] Connection header received
0807 05:07:30.183443 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.183454 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.183512 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.183578 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:30.185166 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.185904 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.187063 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.187286 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.187374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.189772 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:30.189775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:30.189776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:30.189798 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:30.191309 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:30.191762 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:30.191768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:30.191772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:30.191868 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:30.192181 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:30.192183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:30.192184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:30.192361 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:30.192431 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.192925 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.193018 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:07:30.193944 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.181304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:40541)
0807 05:07:30.181721 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:30.181781 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:30.183188 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:30.183781 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:30.183789 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:30.184259 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:30.185001 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.185022 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.186018 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.186021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.186914 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.186924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.187435 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.187438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.187576 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.188097 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:30.188112 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:30.189630 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:30.191404 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:30.191408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:30.191423 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:30.191655 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:30.191965 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:30.191967 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:30.191968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:30.192088 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:30.192454 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:30.192459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:30.192690 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 05:07:30.192882 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 05:07:30.193129 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":110,"threads_started":1}
I20260807 05:07:30.194990 17622 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35931
I20260807 05:07:30.195199 17622 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:30.195680 17622 heartbeater.cc:511] Master 127.1.17.254:35931 requested a full tablet report, sending...
I20260807 05:07:30.196851 17148 ts_manager.cc:194] Registered new tserver with Master: 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751)
I20260807 05:07:30.197573 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:07:30.197597 17148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:40541
I20260807 05:07:30.206035 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.199599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.1.17.254:35931)
0807 05:07:30.199705 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.199708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.199903 (+ 195us) server_negotiation.cc:407] Connection header received
0807 05:07:30.199995 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.199997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.200031 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.200085 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:30.200732 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.201291 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.202115 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.202309 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.202384 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.203615 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:30.203617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:30.203619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:30.203638 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:30.204731 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:30.205127 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:30.205129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:30.205131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:30.205197 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:30.205402 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:30.205404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:30.205405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:30.205579 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:30.205675 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.205843 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.205941 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:07:30.208441 17148 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:07:30.211411 17148 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:07:30.225046 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.219277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.1.17.195:33751)
0807 05:07:30.219676 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.219690 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.220349 (+ 659us) server_negotiation.cc:407] Connection header received
0807 05:07:30.220426 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.220431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.220633 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.220798 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:30.221104 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.222287 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.223713 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.224408 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.224537 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.224668 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.224824 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.224913 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":147,"threads_started":1}
I20260807 05:07:30.225046 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.219170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33751 (local address 127.0.0.1:59094)
0807 05:07:30.220036 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:30.220083 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:30.220207 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:30.220813 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:30.220818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:30.220843 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:30.220982 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.220988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.222466 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.222469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.223598 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.223608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.224543 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.224545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.224620 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.224647 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 05:07:30.224768 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 05:07:30.224900 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":37,"thread_start_us":65,"threads_started":1}
I20260807 05:07:30.225458 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.219826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:42299 (local address 127.0.0.1:60798)
0807 05:07:30.220394 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:30.220407 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:30.220497 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:30.221250 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:30.221253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:30.221270 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:30.221363 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.221369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.222294 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.222301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.223943 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.223948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.225083 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.225087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.225171 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.225194 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:07:30.225237 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 05:07:30.225302 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":177,"thread_start_us":79,"threads_started":1}
I20260807 05:07:30.226177 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.220367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.1.17.194:42299)
0807 05:07:30.220741 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.220753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.220773 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:07:30.220886 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.220891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.221082 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.221272 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:30.221468 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.222166 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.224071 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.224941 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.225722 (+ 781us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.225876 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.225927 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.226026 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":119,"threads_started":1}
I20260807 05:07:30.226725 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.219257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37695 (local address 127.0.0.1:49928)
0807 05:07:30.220103 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:30.220116 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:30.220207 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:30.220978 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:30.220981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:30.220998 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:30.221104 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.221112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.222474 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.222476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.223936 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.223948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.226119 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.226122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.226236 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.226360 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 05:07:30.226455 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 05:07:30.226574 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":207,"thread_start_us":74,"threads_started":1}
I20260807 05:07:30.226775 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.220009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.1.17.193:37695)
0807 05:07:30.220494 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.220509 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.220528 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:07:30.220589 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.220593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.220742 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.220904 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:30.221281 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.222334 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.224746 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.225829 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.225928 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.226055 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.226500 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.226597 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":120,"threads_started":1}
I20260807 05:07:30.228367 17554 tablet_service.cc:1467] Processing CreateTablet for tablet e42dbe4a832146a08e345a76e40ef6e7 (DEFAULT_TABLE table=test-table [id=330e30600d97413caefd15e683fa329d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:30.229547 17554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42dbe4a832146a08e345a76e40ef6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:30.229115 17416 tablet_service.cc:1467] Processing CreateTablet for tablet e42dbe4a832146a08e345a76e40ef6e7 (DEFAULT_TABLE table=test-table [id=330e30600d97413caefd15e683fa329d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:30.229535 17277 tablet_service.cc:1467] Processing CreateTablet for tablet e42dbe4a832146a08e345a76e40ef6e7 (DEFAULT_TABLE table=test-table [id=330e30600d97413caefd15e683fa329d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:30.230137 17416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42dbe4a832146a08e345a76e40ef6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:30.230527 17277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42dbe4a832146a08e345a76e40ef6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:30.235520 17640 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Bootstrap starting.
I20260807 05:07:30.237023 17641 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Bootstrap starting.
I20260807 05:07:30.237767 17642 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Bootstrap starting.
I20260807 05:07:30.237850 17640 tablet_bootstrap.cc:654] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:30.239192 17640 log.cc:824] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:30.239229 17641 tablet_bootstrap.cc:654] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:30.240078 17641 log.cc:824] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:30.240280 17642 tablet_bootstrap.cc:654] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:30.241084 17642 log.cc:824] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:30.241889 17641 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: No bootstrap required, opened a new log
I20260807 05:07:30.241889 17640 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: No bootstrap required, opened a new log
I20260807 05:07:30.242180 17640 ts_tablet_manager.cc:1397] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 05:07:30.242182 17641 ts_tablet_manager.cc:1397] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:07:30.242760 17642 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: No bootstrap required, opened a new log
I20260807 05:07:30.242954 17642 ts_tablet_manager.cc:1397] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:07:30.245509 17642 raft_consensus.cc:348] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.245760 17642 raft_consensus.cc:374] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:30.245831 17642 raft_consensus.cc:729] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be603960c4340359a30369bf3e1c8eb, State: Initialized, Role: FOLLOWER
I20260807 05:07:30.245934 17640 raft_consensus.cc:348] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.246219 17640 raft_consensus.cc:374] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:30.246279 17640 raft_consensus.cc:729] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06a14d884ecc4421a15e3f8a3120becc, State: Initialized, Role: FOLLOWER
I20260807 05:07:30.246187 17641 raft_consensus.cc:348] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.246338 17642 consensus_queue.cc:260] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.246459 17641 raft_consensus.cc:374] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:30.246510 17641 raft_consensus.cc:729] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68369020f360442789263055a3e2d552, State: Initialized, Role: FOLLOWER
I20260807 05:07:30.246706 17640 consensus_queue.cc:260] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.246946 17641 consensus_queue.cc:260] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.247334 17642 ts_tablet_manager.cc:1428] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 05:07:30.247614 17640 ts_tablet_manager.cc:1428] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:07:30.247715 17641 ts_tablet_manager.cc:1428] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:07:30.247725 17483 heartbeater.cc:503] Master 127.1.17.254:35931 was elected leader, sending a full tablet report...
I20260807 05:07:30.247941 17622 heartbeater.cc:503] Master 127.1.17.254:35931 was elected leader, sending a full tablet report...
I20260807 05:07:30.247982 17345 heartbeater.cc:503] Master 127.1.17.254:35931 was elected leader, sending a full tablet report...
W20260807 05:07:30.429831 17623 tablet.cc:2369] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:07:30.446681 17484 tablet.cc:2369] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:07:30.464908 17346 tablet.cc:2369] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:07:30.519469 17647 raft_consensus.cc:482] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:30.519670 17647 raft_consensus.cc:504] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.521026 17647 leader_election.cc:288] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68369020f360442789263055a3e2d552 (127.1.17.193:37695), 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299)
I20260807 05:07:30.525138 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.521403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:48903 (local address 127.1.17.193:37695)
0807 05:07:30.521553 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.521558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.521575 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:07:30.521640 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.521648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.521693 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.521803 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:30.522145 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.522800 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.523952 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.524455 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.524556 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.524597 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.524857 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.524966 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:07:30.525148 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.521200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37695 (local address 127.1.17.195:48903)
0807 05:07:30.521360 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:30.521377 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:30.521475 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:30.521860 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:30.521863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:30.521886 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:30.522014 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.522023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.522914 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.522917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.523808 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.523817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.524584 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.524587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.524698 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.524738 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 05:07:30.524839 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 05:07:30.524938 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1}
I20260807 05:07:30.525307 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.521705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:50763 (local address 127.1.17.194:42299)
0807 05:07:30.521859 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.521863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.522029 (+ 166us) server_negotiation.cc:407] Connection header received
0807 05:07:30.522144 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.522146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.522191 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.522250 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:30.522534 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.523110 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.524201 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.524670 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.524815 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.524867 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.525100 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.525190 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 05:07:30.525669 17649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.521525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:42299 (local address 127.1.17.195:50763)
0807 05:07:30.521942 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:30.521955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:30.522037 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:30.522285 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:30.522288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:30.522304 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:30.522422 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.522428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.523222 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.523224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.524072 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:30.524082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.524824 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.524830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:30.524956 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.524979 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:07:30.525057 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 05:07:30.525181 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":20,"thread_start_us":150,"threads_started":1}
I20260807 05:07:30.525843 17297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42dbe4a832146a08e345a76e40ef6e7" candidate_uuid: "06a14d884ecc4421a15e3f8a3120becc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68369020f360442789263055a3e2d552" is_pre_election: true
I20260807 05:07:30.526168 17297 raft_consensus.cc:2454] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06a14d884ecc4421a15e3f8a3120becc in term 0.
I20260807 05:07:30.526343 17436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42dbe4a832146a08e345a76e40ef6e7" candidate_uuid: "06a14d884ecc4421a15e3f8a3120becc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be603960c4340359a30369bf3e1c8eb" is_pre_election: true
I20260807 05:07:30.526700 17436 raft_consensus.cc:2454] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06a14d884ecc4421a15e3f8a3120becc in term 0.
I20260807 05:07:30.526729 17507 leader_election.cc:302] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06a14d884ecc4421a15e3f8a3120becc, 68369020f360442789263055a3e2d552; no voters:
I20260807 05:07:30.527032 17647 raft_consensus.cc:2781] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:07:30.527128 17647 raft_consensus.cc:482] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:07:30.527187 17647 raft_consensus.cc:3037] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:30.528303 17647 raft_consensus.cc:504] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.528636 17647 leader_election.cc:288] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [CANDIDATE]: Term 1 election: Requested vote from peers 68369020f360442789263055a3e2d552 (127.1.17.193:37695), 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299)
I20260807 05:07:30.529001 17297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42dbe4a832146a08e345a76e40ef6e7" candidate_uuid: "06a14d884ecc4421a15e3f8a3120becc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68369020f360442789263055a3e2d552"
I20260807 05:07:30.529060 17436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42dbe4a832146a08e345a76e40ef6e7" candidate_uuid: "06a14d884ecc4421a15e3f8a3120becc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8be603960c4340359a30369bf3e1c8eb"
I20260807 05:07:30.529106 17297 raft_consensus.cc:3037] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:30.529204 17436 raft_consensus.cc:3037] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:30.530154 17297 raft_consensus.cc:2454] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06a14d884ecc4421a15e3f8a3120becc in term 1.
I20260807 05:07:30.530232 17436 raft_consensus.cc:2454] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06a14d884ecc4421a15e3f8a3120becc in term 1.
I20260807 05:07:30.530436 17507 leader_election.cc:302] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06a14d884ecc4421a15e3f8a3120becc, 68369020f360442789263055a3e2d552; no voters:
I20260807 05:07:30.530624 17647 raft_consensus.cc:2781] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:30.530822 17647 raft_consensus.cc:686] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 1 LEADER]: Becoming Leader. State: Replica: 06a14d884ecc4421a15e3f8a3120becc, State: Running, Role: LEADER
I20260807 05:07:30.531179 17647 consensus_queue.cc:237] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:30.534338 17148 catalog_manager.cc:5495] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc reported cstate change: term changed from 0 to 1, leader changed from <none> to 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195). New cstate: current_term: 1 leader_uuid: "06a14d884ecc4421a15e3f8a3120becc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } health_report { overall_health: UNKNOWN } } }
I20260807 05:07:30.584805 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.581083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59098 (local address 127.1.17.195:33751)
0807 05:07:30.581217 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:30.581220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:30.581288 (+ 68us) server_negotiation.cc:407] Connection header received
0807 05:07:30.581490 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:30.581492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:30.581552 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:30.581710 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:07:30.581927 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.582431 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.583253 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:30.583495 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:30.583571 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:30.583846 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:07:30.584301 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:07:30.584445 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:30.584542 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 05:07:30.584615 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 05:07:30.590978 17297 raft_consensus.cc:1261] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Refusing update from remote peer 06a14d884ecc4421a15e3f8a3120becc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:30.591009 17436 raft_consensus.cc:1261] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 1 FOLLOWER]: Refusing update from remote peer 06a14d884ecc4421a15e3f8a3120becc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:30.591697 17650 consensus_queue.cc:1035] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:30.591879 17647 consensus_queue.cc:1035] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [LEADER]: Connected to new peer: Peer: permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:30.600972 17654 mvcc.cc:205] Tried to move back new op lower bound from 7315780610413457408 to 7315780610177994752. Current Snapshot: MvccSnapshot[applied={T|T < 7315780610413457408}]
I20260807 05:07:30.608630 17148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:30.608819 17148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:30.611112 17148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Sending DeleteTablet for 3 replicas of tablet e42dbe4a832146a08e345a76e40ef6e7
I20260807 05:07:30.611856 17554 tablet_service.cc:1514] Processing DeleteTablet for tablet e42dbe4a832146a08e345a76e40ef6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:30 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:59094
I20260807 05:07:30.611915 17416 tablet_service.cc:1514] Processing DeleteTablet for tablet e42dbe4a832146a08e345a76e40ef6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:30 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:60798
I20260807 05:07:30.612365 17663 tablet_replica.cc:330] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: stopping tablet replica
I20260807 05:07:30.612399 17277 tablet_service.cc:1514] Processing DeleteTablet for tablet e42dbe4a832146a08e345a76e40ef6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:30 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:49928
I20260807 05:07:30.612454 17664 tablet_replica.cc:330] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: stopping tablet replica
I20260807 05:07:30.612756 17664 raft_consensus.cc:2229] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:07:30.612807 17663 raft_consensus.cc:2229] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:07:30.613025 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 17212
I20260807 05:07:30.613812 17663 raft_consensus.cc:2258] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:30.613122 17664 raft_consensus.cc:2258] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:30.616701 17663 ts_tablet_manager.cc:1905] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:30.618114 17664 ts_tablet_manager.cc:1905] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:30.619598 17663 ts_tablet_manager.cc:1918] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:30.619712 17663 log.cc:1196] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/e42dbe4a832146a08e345a76e40ef6e7
I20260807 05:07:30.620057 17663 ts_tablet_manager.cc:1939] T e42dbe4a832146a08e345a76e40ef6e7 P 06a14d884ecc4421a15e3f8a3120becc: Deleting consensus metadata
I20260807 05:07:30.621094 17664 ts_tablet_manager.cc:1918] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:30.621260 17664 log.cc:1196] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/e42dbe4a832146a08e345a76e40ef6e7
I20260807 05:07:30.621366 17133 catalog_manager.cc:4841] TS 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751): tablet e42dbe4a832146a08e345a76e40ef6e7 (table test-table [id=330e30600d97413caefd15e683fa329d]) successfully deleted
I20260807 05:07:30.621701 17664 ts_tablet_manager.cc:1939] T e42dbe4a832146a08e345a76e40ef6e7 P 8be603960c4340359a30369bf3e1c8eb: Deleting consensus metadata
I20260807 05:07:30.622771 17135 catalog_manager.cc:4841] TS 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299): tablet e42dbe4a832146a08e345a76e40ef6e7 (table test-table [id=330e30600d97413caefd15e683fa329d]) successfully deleted
W20260807 05:07:30.624275 17135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:37695 (error 108)
I20260807 05:07:30.625166 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 17351
I20260807 05:07:30.625332 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:30.625037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37695 (local address 127.0.0.1:49940)
0807 05:07:30.625182 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:30.625254 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:37695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 05:07:30.630306 17135 catalog_manager.cc:4584] TS 68369020f360442789263055a3e2d552 (127.1.17.193:37695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e42dbe4a832146a08e345a76e40ef6e7: Network error: Client connection negotiation failed: client connection to 127.1.17.193:37695: connect: Connection refused (error 111)
I20260807 05:07:30.634461 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 17488
I20260807 05:07:30.641479 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 17115
I20260807 05:07:50.649382 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35931
--webserver_interface=127.1.17.254
--webserver_port=36225
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:07:50.764751 17668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:50.765071 17668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:50.765162 17668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:50.769138 17668 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 05:07:50.769207 17668 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 05:07:50.769236 17668 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 05:07:50.769279 17668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:07:50.769317 17668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:50.769382 17668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:50.769424 17668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:07:50.769461 17668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:07:50.774415 17668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=36225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:50.775490 17668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:50.776539 17668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:50.791230 17674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:50.791231 17675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:50.791292 17668 server_base.cc:1034] running on GCE node
W20260807 05:07:50.791231 17677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:50.792127 17668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:50.793331 17668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:50.794508 17668 hybrid_clock.cc:648] HybridClock initialized: now 1786079270794489 us; error 29 us; skew 500 ppm
Aug 07 05:07:50 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079270, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:50.801985 17668 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:07:50.802901 17668 webserver.cc:460] Webserver started at http://127.1.17.254:36225/ using document root <none> and password file <none>
I20260807 05:07:50.803462 17668 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:50.803531 17668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:50.807945 17668 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 05:07:50.810806 17684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:50.812649 17668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:07:50.812829 17668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "c10bc2c1d41748deb8e794750be53354"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "099a91ad7ea7b3db29f1beb4b645565a"
server_key_iv: "efcad6dc00f59b3feb5f8a4b532808f7"
server_key_version: "encryptionkey@0"
I20260807 05:07:50.813759 17668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:50.833076 17668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:50.841063 17668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:50.841393 17668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:50.848637 17668 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:35931
I20260807 05:07:50.848999 17738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:35931 every 8 connection(s)
I20260807 05:07:50.849870 17668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:07:50.854509 17739 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:07:50.856159 17739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Bootstrap starting.
I20260807 05:07:50.857090 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 17668
I20260807 05:07:50.857643 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:37695
--local_ip_for_outbound_sockets=127.1.17.193
--tserver_master_addrs=127.1.17.254:35931
--webserver_port=42551
--webserver_interface=127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:07:50.880753 17739 log.cc:824] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:50.892302 17739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:07:50.892762 17739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Bootstrap complete.
I20260807 05:07:50.898655 17739 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:50.899194 17739 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10bc2c1d41748deb8e794750be53354, State: Initialized, Role: FOLLOWER
I20260807 05:07:50.899762 17739 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:50.899926 17739 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:07:50.899999 17739 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:07:50.900096 17739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 1 FOLLOWER]: Advancing to term 2
I20260807 05:07:50.902670 17739 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:50.903080 17739 leader_election.cc:302] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10bc2c1d41748deb8e794750be53354; no voters:
I20260807 05:07:50.903494 17739 leader_election.cc:288] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 05:07:50.903644 17743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 05:07:50.903944 17743 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 LEADER]: Becoming Leader. State: Replica: c10bc2c1d41748deb8e794750be53354, State: Running, Role: LEADER
I20260807 05:07:50.904263 17743 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:07:50.904891 17739 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:07:50.905959 17745 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10bc2c1d41748deb8e794750be53354. Latest consensus state: current_term: 2 leader_uuid: "c10bc2c1d41748deb8e794750be53354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } } }
I20260807 05:07:50.905977 17744 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c10bc2c1d41748deb8e794750be53354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } } }
I20260807 05:07:50.906111 17745 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:50.906111 17744 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: This master's current role is: LEADER
I20260807 05:07:50.906498 17752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:07:50.911342 17752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:07:50.911989 17752 catalog_manager.cc:1259] Loaded cluster ID: 9e1c73cd8da244d2b960ccde28abe0dd
I20260807 05:07:50.912053 17752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:07:50.914609 17752 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:07:50.915321 17752 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Loaded TSK: 0
I20260807 05:07:50.934342 17752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Generated new TSK 1
I20260807 05:07:50.934621 17752 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:07:50.990705 17741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:50.990983 17741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:50.991060 17741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:50.995069 17741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:50.995170 17741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:50.995257 17741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:07:51.000459 17741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:37695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=42551
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:51.001487 17741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:51.002573 17741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:51.013813 17770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:51.013813 17767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:51.013861 17768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:51.014281 17741 server_base.cc:1034] running on GCE node
I20260807 05:07:51.014717 17741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:51.015548 17741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:51.016757 17741 hybrid_clock.cc:648] HybridClock initialized: now 1786079271016726 us; error 43 us; skew 500 ppm
Aug 07 05:07:51 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079271, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:51.020265 17741 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:07:51.021270 17741 webserver.cc:460] Webserver started at http://127.1.17.193:42551/ using document root <none> and password file <none>
I20260807 05:07:51.021845 17741 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:51.021930 17741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:51.026027 17741 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:07:51.028510 17777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:51.029889 17741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:07:51.030035 17741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "68369020f360442789263055a3e2d552"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "dc23ed44d48b1257987c9dbcd2b44814"
server_key_iv: "fc53c65160376f967b16e6a3a340f305"
server_key_version: "encryptionkey@0"
I20260807 05:07:51.030607 17741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:51.061721 17741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:51.065179 17741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:51.065428 17741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:51.066159 17741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:51.067322 17786 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:07:51.071574 17741 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:07:51.071663 17741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 05:07:51.071764 17741 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:07:51.073586 17741 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:07:51.073663 17741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 05:07:51.073979 17786 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Bootstrap starting.
I20260807 05:07:51.085444 17741 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:37695
I20260807 05:07:51.085500 17898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:37695 every 8 connection(s)
I20260807 05:07:51.086470 17741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:07:51.093286 17786 log.cc:824] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:51.096572 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 17741
I20260807 05:07:51.097163 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:42299
--local_ip_for_outbound_sockets=127.1.17.194
--tserver_master_addrs=127.1.17.254:35931
--webserver_port=38861
--webserver_interface=127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:07:51 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079271, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:51.103915 17786 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:07:51.104400 17786 tablet_bootstrap.cc:492] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Bootstrap complete.
I20260807 05:07:51.104854 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.088548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:42415)
0807 05:07:51.089054 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.089109 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.089936 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.091345 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.091351 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.091693 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:51.092158 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.092171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.094023 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.094030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.094978 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.094989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.095993 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.095996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.096143 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.096909 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:51.096935 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:51.098944 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:51.101356 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.101361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.101375 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.101660 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.102023 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.102027 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.102029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.102175 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.102843 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:51.102850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:51.103169 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.103397 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.103773 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":144,"threads_started":1}
I20260807 05:07:51.106277 17786 ts_tablet_manager.cc:1397] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Time spent bootstrapping tablet: real 0.033s user 0.018s sys 0.011s
I20260807 05:07:51.106446 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.088581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:42415 (local address 127.1.17.254:35931)
0807 05:07:51.089199 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.089227 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.089282 (+ 55us) server_negotiation.cc:407] Connection header received
0807 05:07:51.090394 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.090411 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.090881 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.091305 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:51.092339 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.093850 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.095136 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.095841 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.096046 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.099204 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:51.099213 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:51.099233 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:51.099276 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:51.101224 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.101794 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.101798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.101802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.101919 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.102318 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.102322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.102325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.102716 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:51.102865 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.105345 (+ 2480us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.105708 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":149,"threads_started":1}
I20260807 05:07:51.108640 17899 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35931
I20260807 05:07:51.108898 17899 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:51.109395 17899 heartbeater.cc:511] Master 127.1.17.254:35931 requested a full tablet report, sending...
I20260807 05:07:51.109949 17786 raft_consensus.cc:348] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:51.110620 17786 raft_consensus.cc:729] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68369020f360442789263055a3e2d552, State: Initialized, Role: FOLLOWER
I20260807 05:07:51.111157 17786 consensus_queue.cc:260] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } } peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } }
I20260807 05:07:51.111827 17702 ts_manager.cc:194] Registered new tserver with Master: 68369020f360442789263055a3e2d552 (127.1.17.193:37695)
I20260807 05:07:51.112320 17786 ts_tablet_manager.cc:1428] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:07:51.115263 17702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:42415
I20260807 05:07:51.117821 17899 heartbeater.cc:503] Master 127.1.17.254:35931 was elected leader, sending a full tablet report...
Aug 07 05:07:51 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079270, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.193@KRBTEST.COM
I20260807 05:07:51.126086 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.114797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.1.17.193:37695)
0807 05:07:51.115137 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.115151 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.115650 (+ 499us) server_negotiation.cc:407] Connection header received
0807 05:07:51.115830 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.115835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.116030 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.116164 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:51.116895 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.117934 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.119418 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.119800 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.119885 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.122360 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:51.122369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:51.122379 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:51.122400 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:51.124143 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.124724 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.124730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.124732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.124879 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.125169 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.125172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.125173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.125387 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:51.125489 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.125832 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.125957 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":99,"threads_started":1}
I20260807 05:07:51.126081 17906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.114703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37695 (local address 127.0.0.1:35436)
0807 05:07:51.115500 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.115557 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.115688 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.116177 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.116183 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.116591 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:51.116745 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.116752 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.118057 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.118061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.119223 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.119236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.119912 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.119916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.120030 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.120555 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:51.120569 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:51.122208 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:51.124278 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.124289 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.124294 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.124596 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.124982 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.124985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.124987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.125081 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.125499 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:51.125504 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:51.125620 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.125762 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.125927 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":115,"threads_started":1}
I20260807 05:07:51.127082 17830 tablet_service.cc:1514] Processing DeleteTablet for tablet e42dbe4a832146a08e345a76e40ef6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:30 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:35436
I20260807 05:07:51.127903 17908 tablet_replica.cc:330] stopping tablet replica
I20260807 05:07:51.128192 17908 raft_consensus.cc:2229] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:07:51.128432 17908 raft_consensus.cc:2258] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:51.129374 17908 ts_tablet_manager.cc:1905] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:51.132855 17908 ts_tablet_manager.cc:1918] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:51.132974 17908 log.cc:1196] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal/wals/e42dbe4a832146a08e345a76e40ef6e7
I20260807 05:07:51.133249 17908 ts_tablet_manager.cc:1939] T e42dbe4a832146a08e345a76e40ef6e7 P 68369020f360442789263055a3e2d552: Deleting consensus metadata
I20260807 05:07:51.134913 17688 catalog_manager.cc:4841] TS 68369020f360442789263055a3e2d552 (127.1.17.193:37695): tablet e42dbe4a832146a08e345a76e40ef6e7 (table test-table [id=330e30600d97413caefd15e683fa329d]) successfully deleted
W20260807 05:07:51.215632 17904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:51.215844 17904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:51.215883 17904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:51.219895 17904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:51.219951 17904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:51.220006 17904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:07:51.225106 17904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:42299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=38861
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:51.225958 17904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:51.226830 17904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:51.236220 17916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:51.236220 17913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:51.236501 17914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:51.236501 17904 server_base.cc:1034] running on GCE node
I20260807 05:07:51.237061 17904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:51.237658 17904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:51.238901 17904 hybrid_clock.cc:648] HybridClock initialized: now 1786079271238841 us; error 44 us; skew 500 ppm
Aug 07 05:07:51 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079271, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:51.241926 17904 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:07:51.242815 17904 webserver.cc:460] Webserver started at http://127.1.17.194:38861/ using document root <none> and password file <none>
I20260807 05:07:51.243404 17904 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:51.243525 17904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:51.247383 17904 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:07:51.249576 17924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:51.250725 17904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:07:51.250867 17904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "8be603960c4340359a30369bf3e1c8eb"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "39a5f06c43293e428489007811887e0b"
server_key_iv: "f786ea9331f0f510ac916d019f18b73b"
server_key_version: "encryptionkey@0"
I20260807 05:07:51.251392 17904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:51.262629 17904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:51.265704 17904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:51.265904 17904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:51.266449 17904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:51.267217 17904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:51.267292 17904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:51.267361 17904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:51.267395 17904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:51.276047 17904 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:42299
I20260807 05:07:51.276074 18040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:42299 every 8 connection(s)
I20260807 05:07:51.277045 17904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:07:51.285575 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 17904
I20260807 05:07:51.286156 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:33751
--local_ip_for_outbound_sockets=127.1.17.195
--tserver_master_addrs=127.1.17.254:35931
--webserver_port=44717
--webserver_interface=127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 05:07:51 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079271, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:51.290359 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.278761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:42019 (local address 127.1.17.254:35931)
0807 05:07:51.278909 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.278914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.279783 (+ 869us) server_negotiation.cc:407] Connection header received
0807 05:07:51.281122 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.281125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.281171 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.281251 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:51.282353 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.282882 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.283964 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.284187 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.284265 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.286839 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:51.286842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:51.286844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:51.286867 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:51.288248 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.288746 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.288750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.288752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.288887 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.289233 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.289237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.289239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.289420 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:51.289532 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.290045 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.290185 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:07:51.291203 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.279196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:42019)
0807 05:07:51.279626 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.279667 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.280903 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.281415 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.281421 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.281758 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:51.282214 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.282227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.282990 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.282992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.283849 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.283857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.284315 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.284318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.284463 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.285056 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:51.285072 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:51.286669 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:51.288370 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.288375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.288390 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.288643 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.289000 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.289003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.289005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.289140 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.289543 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:51.289551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:51.289808 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.290004 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.290316 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260807 05:07:51.292392 18041 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35931
I20260807 05:07:51.292642 18041 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:51.293159 18041 heartbeater.cc:511] Master 127.1.17.254:35931 requested a full tablet report, sending...
I20260807 05:07:51.294296 17702 ts_manager.cc:194] Registered new tserver with Master: 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299)
I20260807 05:07:51.295153 17702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:42019
W20260807 05:07:51.397506 18044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:07:51.397763 18044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:07:51.397848 18044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:07:51.401906 18044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:07:51.401978 18044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:07:51.402036 18044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:07:51.407738 18044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:33751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=44717
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:07:51.408792 18044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:07:51.409909 18044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:07:51.420008 18053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:51.420017 18050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:07:51.420020 18051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:07:51.420255 18044 server_base.cc:1034] running on GCE node
I20260807 05:07:51.420732 18044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:07:51.421463 18044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:07:51.422631 18044 hybrid_clock.cc:648] HybridClock initialized: now 1786079271422596 us; error 43 us; skew 500 ppm
Aug 07 05:07:51 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079271, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:07:51.425479 18044 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:07:51.426354 18044 webserver.cc:460] Webserver started at http://127.1.17.195:44717/ using document root <none> and password file <none>
I20260807 05:07:51.426939 18044 fs_manager.cc:359] Metadata directory not provided
I20260807 05:07:51.427055 18044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:07:51.430956 18044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:07:51.433166 18060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:51.434553 18044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:07:51.434685 18044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "06a14d884ecc4421a15e3f8a3120becc"
format_stamp: "Formatted at 2026-08-07 05:07:30 on dist-test-slave-1x4g"
server_key: "caf4ffd83a5075d78eb1c9a3f86db1f1"
server_key_iv: "3b7d3f4b2964a89d4d0188862ead3186"
server_key_version: "encryptionkey@0"
I20260807 05:07:51.435257 18044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:07:51.456555 18044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:07:51.459956 18044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:07:51.460166 18044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:07:51.460878 18044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:07:51.461793 18044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:07:51.461864 18044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:51.461930 18044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:07:51.461972 18044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:07:51.470892 18044 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:33751
I20260807 05:07:51.470952 18178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:33751 every 8 connection(s)
I20260807 05:07:51.471930 18044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:07:51.473842 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 18044
Aug 07 05:07:51 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079271, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:07:51.483379 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.474522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.1.17.254:35931)
0807 05:07:51.474751 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.474757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.474932 (+ 175us) server_negotiation.cc:407] Connection header received
0807 05:07:51.475256 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.475259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.475314 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.475411 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:51.476121 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.476984 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.477833 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.478218 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.478335 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.479715 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:51.479718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:51.479721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:51.479748 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:51.481457 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.481970 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.481974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.481976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.482075 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.482358 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.482361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.482363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.482624 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:51.482788 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.482994 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.483151 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260807 05:07:51.486939 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.473874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:55401 (local address 127.1.17.254:35931)
0807 05:07:51.474029 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.474033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.475217 (+ 1184us) server_negotiation.cc:407] Connection header received
0807 05:07:51.476727 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.476729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.476771 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.476881 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:51.478175 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.478684 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.479936 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.480132 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.480246 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.483189 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:51.483193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:51.483196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:51.483218 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:51.484893 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.485525 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.485527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.485529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.485601 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.485990 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.485992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.485994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.486157 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:51.486241 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.486739 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.486828 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":71}
I20260807 05:07:51.487856 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.474428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:55401)
0807 05:07:51.475048 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.475107 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.476488 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.477060 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.477069 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.477436 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:07:51.478033 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.478051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.478804 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.478810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.479816 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.479826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.480290 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.480306 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.480480 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.481103 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:07:51.481119 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:07:51.483005 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:07:51.485002 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.485007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.485018 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.485413 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.485728 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:07:51.485731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:07:51.485733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:07:51.485870 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:07:51.486254 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:07:51.486259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:07:51.486509 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.486674 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.486946 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":156,"threads_started":1}
I20260807 05:07:51.488966 18179 heartbeater.cc:347] Connected to a master server at 127.1.17.254:35931
I20260807 05:07:51.489214 18179 heartbeater.cc:464] Registering TS with master...
I20260807 05:07:51.489744 18179 heartbeater.cc:511] Master 127.1.17.254:35931 requested a full tablet report, sending...
I20260807 05:07:51.490813 17702 ts_manager.cc:194] Registered new tserver with Master: 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751)
I20260807 05:07:51.491384 17702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:55401
I20260807 05:07:51.497225 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:07:51.501124 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.497839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59672 (local address 127.1.17.254:35931)
0807 05:07:51.497955 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.497958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.498214 (+ 256us) server_negotiation.cc:407] Connection header received
0807 05:07:51.498380 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.498382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.498484 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.498568 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:07:51.498782 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.499332 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.500132 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.500328 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.500398 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.500653 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:07:51.500821 (+ 168us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 05:07:51.501031 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59672: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 05:07:51.501200 17902 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59672: token expired
W20260807 05:07:51.501561 17625 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.17.254:35931)
I20260807 05:07:51.508572 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.501907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.1.17.254:35931)
0807 05:07:51.502011 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.502014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.502024 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:07:51.502117 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.502119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.502150 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.502209 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:07:51.502794 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.503302 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.504015 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.504209 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.504273 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.505636 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:07:51.505639 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:07:51.505641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:07:51.505661 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:07:51.507285 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.507648 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.507651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.507652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.507728 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:07:51.507971 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:07:51.507973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:07:51.507974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:07:51.508122 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:07:51.508220 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.508378 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.508469 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:07:51.510867 17702 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:07:51.514216 17702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:07:51.521919 17830 tablet_service.cc:1467] Processing CreateTablet for tablet 8982fb7467244861a8c8c2697aa95e85 (DEFAULT_TABLE table=test-table [id=de7bc4d7acad4fdfae013c323d422e23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:51.522579 17830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8982fb7467244861a8c8c2697aa95e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:51.525315 17786 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Bootstrap starting.
I20260807 05:07:51.526198 17786 tablet_bootstrap.cc:654] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:51.526563 18189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.521290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33751 (local address 127.0.0.1:35124)
0807 05:07:51.521767 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.521780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.521864 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.522611 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.522614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.522635 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:51.522724 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.522732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.524073 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.524077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.525013 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.525023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.526143 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.526146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.526229 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.526271 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.526354 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.526422 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":24,"thread_start_us":123,"threads_started":1}
I20260807 05:07:51.526620 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.521734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.1.17.195:33751)
0807 05:07:51.522126 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.522139 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.522175 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:07:51.522249 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.522256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.522466 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.522609 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:51.522855 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.523980 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.525134 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.526010 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.526130 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.526281 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.526387 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.526486 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":86,"threads_started":1}
I20260807 05:07:51.527035 17906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.520838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:42299 (local address 127.0.0.1:46622)
0807 05:07:51.520976 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.520993 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.521072 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.522056 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.522059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.522077 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:51.522181 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.522187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.524044 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.524048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.525301 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.525313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.526636 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.526643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.526752 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.526774 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.526826 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.526922 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":32}
I20260807 05:07:51.528203 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.521059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46622 (local address 127.1.17.194:42299)
0807 05:07:51.521543 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.521553 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.521573 (+ 20us) server_negotiation.cc:407] Connection header received
0807 05:07:51.521666 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.521672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.521900 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.522081 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:51.522276 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.523897 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.525534 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.526463 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.527797 (+ 1334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.527939 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.527987 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.528068 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":165,"threads_started":1}
I20260807 05:07:51.529688 18109 tablet_service.cc:1467] Processing CreateTablet for tablet 8982fb7467244861a8c8c2697aa95e85 (DEFAULT_TABLE table=test-table [id=de7bc4d7acad4fdfae013c323d422e23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:51.530650 18109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8982fb7467244861a8c8c2697aa95e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:51.530520 17972 tablet_service.cc:1467] Processing CreateTablet for tablet 8982fb7467244861a8c8c2697aa95e85 (DEFAULT_TABLE table=test-table [id=de7bc4d7acad4fdfae013c323d422e23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:07:51.531416 17972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8982fb7467244861a8c8c2697aa95e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:07:51.533999 17786 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: No bootstrap required, opened a new log
I20260807 05:07:51.534101 17786 ts_tablet_manager.cc:1397] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260807 05:07:51.534610 17786 raft_consensus.cc:348] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.534730 17786 raft_consensus.cc:374] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:51.534767 17786 raft_consensus.cc:729] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68369020f360442789263055a3e2d552, State: Initialized, Role: FOLLOWER
I20260807 05:07:51.534937 17786 consensus_queue.cc:260] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.535271 17786 ts_tablet_manager.cc:1428] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 05:07:51.538651 18192 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Bootstrap starting.
I20260807 05:07:51.540102 18193 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Bootstrap starting.
I20260807 05:07:51.541390 18192 tablet_bootstrap.cc:654] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:51.542168 18193 tablet_bootstrap.cc:654] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Neither blocks nor log segments found. Creating new log.
I20260807 05:07:51.542167 18192 log.cc:824] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:51.542989 18193 log.cc:824] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Log is configured to *not* fsync() on all Append() calls
I20260807 05:07:51.544096 18192 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: No bootstrap required, opened a new log
I20260807 05:07:51.544291 18192 ts_tablet_manager.cc:1397] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:07:51.544715 18193 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: No bootstrap required, opened a new log
I20260807 05:07:51.544924 18193 ts_tablet_manager.cc:1397] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 05:07:51.547016 18192 raft_consensus.cc:348] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.547242 18192 raft_consensus.cc:374] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:51.547320 18192 raft_consensus.cc:729] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06a14d884ecc4421a15e3f8a3120becc, State: Initialized, Role: FOLLOWER
I20260807 05:07:51.547477 18193 raft_consensus.cc:348] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.547737 18193 raft_consensus.cc:374] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:07:51.547814 18193 raft_consensus.cc:729] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be603960c4340359a30369bf3e1c8eb, State: Initialized, Role: FOLLOWER
I20260807 05:07:51.547787 18192 consensus_queue.cc:260] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.548264 18193 consensus_queue.cc:260] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.548441 18179 heartbeater.cc:503] Master 127.1.17.254:35931 was elected leader, sending a full tablet report...
I20260807 05:07:51.548823 18192 ts_tablet_manager.cc:1428] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Time spent starting tablet: real 0.004s user 0.005s sys 0.001s
I20260807 05:07:51.549207 18193 ts_tablet_manager.cc:1428] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260807 05:07:51.549512 18041 heartbeater.cc:503] Master 127.1.17.254:35931 was elected leader, sending a full tablet report...
W20260807 05:07:51.587662 17900 tablet.cc:2369] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:07:51.675055 18197 raft_consensus.cc:482] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:07:51.675282 18197 raft_consensus.cc:504] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.676509 18197 leader_election.cc:288] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751), 68369020f360442789263055a3e2d552 (127.1.17.193:37695)
I20260807 05:07:51.680429 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.676908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:53571 (local address 127.1.17.195:33751)
0807 05:07:51.677060 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.677065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.677327 (+ 262us) server_negotiation.cc:407] Connection header received
0807 05:07:51.677463 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.677469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.677525 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.677604 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:51.677916 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.678391 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.679483 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.679858 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.679947 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.680013 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.680210 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.680297 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:07:51.680464 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.676743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37695 (local address 127.1.17.194:36117)
0807 05:07:51.676874 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.676892 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.676997 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.677503 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.677506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.677525 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:51.677668 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.677678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.678494 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.678499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.679329 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.679337 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.680058 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.680061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.680184 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.680207 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.680281 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.680342 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":7}
I20260807 05:07:51.680473 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.676743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33751 (local address 127.1.17.194:53571)
0807 05:07:51.677220 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.677235 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:07:51.677341 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:07:51.677644 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:07:51.677647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:07:51.677662 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:07:51.677801 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.677807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.678537 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.678540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.679327 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:07:51.679336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.679985 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.679992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:07:51.680087 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.680109 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:07:51.680183 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:07:51.680286 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":137,"thread_start_us":116,"threads_started":1}
I20260807 05:07:51.681043 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.676930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:36117 (local address 127.1.17.193:37695)
0807 05:07:51.677202 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.677207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.677220 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:07:51.677285 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.677288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.677365 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.677483 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:07:51.677796 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.678375 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.679456 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.679931 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.680051 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.680209 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.680307 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.680392 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
I20260807 05:07:51.681237 18130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8982fb7467244861a8c8c2697aa95e85" candidate_uuid: "8be603960c4340359a30369bf3e1c8eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06a14d884ecc4421a15e3f8a3120becc" is_pre_election: true
I20260807 05:07:51.681578 18130 raft_consensus.cc:2454] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8be603960c4340359a30369bf3e1c8eb in term 0.
I20260807 05:07:51.681540 17851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8982fb7467244861a8c8c2697aa95e85" candidate_uuid: "8be603960c4340359a30369bf3e1c8eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68369020f360442789263055a3e2d552" is_pre_election: true
I20260807 05:07:51.681813 17851 raft_consensus.cc:2454] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8be603960c4340359a30369bf3e1c8eb in term 0.
I20260807 05:07:51.682109 17926 leader_election.cc:302] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06a14d884ecc4421a15e3f8a3120becc, 8be603960c4340359a30369bf3e1c8eb; no voters:
I20260807 05:07:51.682422 18197 raft_consensus.cc:2781] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:07:51.682511 18197 raft_consensus.cc:482] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:07:51.682554 18197 raft_consensus.cc:3037] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:51.683598 18197 raft_consensus.cc:504] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.683936 18197 leader_election.cc:288] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [CANDIDATE]: Term 1 election: Requested vote from peers 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751), 68369020f360442789263055a3e2d552 (127.1.17.193:37695)
I20260807 05:07:51.684230 18130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8982fb7467244861a8c8c2697aa95e85" candidate_uuid: "8be603960c4340359a30369bf3e1c8eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06a14d884ecc4421a15e3f8a3120becc"
I20260807 05:07:51.684271 17851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8982fb7467244861a8c8c2697aa95e85" candidate_uuid: "8be603960c4340359a30369bf3e1c8eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68369020f360442789263055a3e2d552"
I20260807 05:07:51.684360 18130 raft_consensus.cc:3037] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:51.684383 17851 raft_consensus.cc:3037] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:07:51.685660 18130 raft_consensus.cc:2454] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8be603960c4340359a30369bf3e1c8eb in term 1.
I20260807 05:07:51.685660 17851 raft_consensus.cc:2454] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8be603960c4340359a30369bf3e1c8eb in term 1.
I20260807 05:07:51.685969 17928 leader_election.cc:302] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68369020f360442789263055a3e2d552, 8be603960c4340359a30369bf3e1c8eb; no voters:
I20260807 05:07:51.686156 18197 raft_consensus.cc:2781] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:07:51.686357 18197 raft_consensus.cc:686] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 1 LEADER]: Becoming Leader. State: Replica: 8be603960c4340359a30369bf3e1c8eb, State: Running, Role: LEADER
I20260807 05:07:51.686684 18197 consensus_queue.cc:237] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:07:51.689774 17702 catalog_manager.cc:5495] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8be603960c4340359a30369bf3e1c8eb (127.1.17.194). New cstate: current_term: 1 leader_uuid: "8be603960c4340359a30369bf3e1c8eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } health_report { overall_health: UNKNOWN } } }
I20260807 05:07:51.717885 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.714583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.1.17.194:42299)
0807 05:07:51.714716 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:07:51.714719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:07:51.714731 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:07:51.714806 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:07:51.714807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:07:51.714860 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:07:51.714964 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:07:51.715162 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.715653 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.716446 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:07:51.716660 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:07:51.716743 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:07:51.717060 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:07:51.717506 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:07:51.717598 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:07:51.717720 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 05:07:51.717766 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 05:07:51.722880 18180 tablet.cc:2369] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:07:51.723735 18130 raft_consensus.cc:1261] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 1 FOLLOWER]: Refusing update from remote peer 8be603960c4340359a30369bf3e1c8eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:51.723798 17851 raft_consensus.cc:1261] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Refusing update from remote peer 8be603960c4340359a30369bf3e1c8eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:07:51.724464 18197 consensus_queue.cc:1035] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:51.724653 18200 consensus_queue.cc:1035] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:07:51.730374 18204 mvcc.cc:205] Tried to move back new op lower bound from 7315780696973037568 to 7315780696831279104. Current Snapshot: MvccSnapshot[applied={T|T < 7315780696973037568}]
I20260807 05:07:51.732832 18205 mvcc.cc:205] Tried to move back new op lower bound from 7315780696973037568 to 7315780696831279104. Current Snapshot: MvccSnapshot[applied={T|T < 7315780696973037568}]
I20260807 05:07:51.739600 17701 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:51.739760 17701 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:07:51.742131 17701 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Sending DeleteTablet for 3 replicas of tablet 8982fb7467244861a8c8c2697aa95e85
I20260807 05:07:51.742877 17972 tablet_service.cc:1514] Processing DeleteTablet for tablet 8982fb7467244861a8c8c2697aa95e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:51 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:46622
I20260807 05:07:51.742919 18109 tablet_service.cc:1514] Processing DeleteTablet for tablet 8982fb7467244861a8c8c2697aa95e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:51 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:35124
I20260807 05:07:51.743371 18212 tablet_replica.cc:330] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: stopping tablet replica
I20260807 05:07:51.743363 17830 tablet_service.cc:1514] Processing DeleteTablet for tablet 8982fb7467244861a8c8c2697aa95e85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:07:51 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:35436
I20260807 05:07:51.743391 18213 tablet_replica.cc:330] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: stopping tablet replica
I20260807 05:07:51.743685 18213 raft_consensus.cc:2229] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:07:51.743742 18212 raft_consensus.cc:2229] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:07:51.743818 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 17741
I20260807 05:07:51.744010 18213 raft_consensus.cc:2258] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:51.744287 18212 raft_consensus.cc:2258] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:07:51.745587 18213 ts_tablet_manager.cc:1905] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:51.746220 18212 ts_tablet_manager.cc:1905] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:07:51.748337 18213 ts_tablet_manager.cc:1918] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:51.748437 18213 log.cc:1196] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/8982fb7467244861a8c8c2697aa95e85
I20260807 05:07:51.748819 18213 ts_tablet_manager.cc:1939] T 8982fb7467244861a8c8c2697aa95e85 P 06a14d884ecc4421a15e3f8a3120becc: Deleting consensus metadata
I20260807 05:07:51.750007 18212 ts_tablet_manager.cc:1918] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:07:51.750174 18212 log.cc:1196] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/8982fb7467244861a8c8c2697aa95e85
I20260807 05:07:51.750203 17686 catalog_manager.cc:4841] TS 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751): tablet 8982fb7467244861a8c8c2697aa95e85 (table test-table [id=de7bc4d7acad4fdfae013c323d422e23]) successfully deleted
I20260807 05:07:51.750639 18212 ts_tablet_manager.cc:1939] T 8982fb7467244861a8c8c2697aa95e85 P 8be603960c4340359a30369bf3e1c8eb: Deleting consensus metadata
I20260807 05:07:51.751760 17688 catalog_manager.cc:4841] TS 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299): tablet 8982fb7467244861a8c8c2697aa95e85 (table test-table [id=de7bc4d7acad4fdfae013c323d422e23]) successfully deleted
W20260807 05:07:51.755750 17688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:37695 (error 108)
I20260807 05:07:51.756420 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 17904
I20260807 05:07:51.757380 17906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:07:51.756519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:37695 (local address 127.0.0.1:35448)
0807 05:07:51.757191 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 05:07:51.757276 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:37695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567}
W20260807 05:07:51.757755 17688 catalog_manager.cc:4584] TS 68369020f360442789263055a3e2d552 (127.1.17.193:37695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8982fb7467244861a8c8c2697aa95e85: Network error: Client connection negotiation failed: client connection to 127.1.17.193:37695: connect: Connection refused (error 111)
I20260807 05:07:51.763629 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 18044
I20260807 05:07:51.769919 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 17668
I20260807 05:08:11.777921 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35931
--webserver_interface=127.1.17.254
--webserver_port=36225
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:11.887705 18216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:11.887990 18216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:11.888059 18216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:11.891943 18216 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 05:08:11.892014 18216 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 05:08:11.892041 18216 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 05:08:11.892072 18216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:11.892110 18216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:11.892153 18216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:11.892187 18216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:11.892223 18216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:11.897197 18216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=36225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:11.898249 18216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:11.899267 18216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:11.907757 18224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:11.907776 18221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:11.907768 18222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:11.907816 18216 server_base.cc:1034] running on GCE node
I20260807 05:08:11.908591 18216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:11.909693 18216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:11.910856 18216 hybrid_clock.cc:648] HybridClock initialized: now 1786079291910837 us; error 30 us; skew 500 ppm
Aug 07 05:08:11 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079291, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:11.913698 18216 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:08:11.914587 18216 webserver.cc:460] Webserver started at http://127.1.17.254:36225/ using document root <none> and password file <none>
I20260807 05:08:11.915141 18216 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:11.915200 18216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:11.918901 18216 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:08:11.921020 18232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:11.922192 18216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:11.922317 18216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "c10bc2c1d41748deb8e794750be53354"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "099a91ad7ea7b3db29f1beb4b645565a"
server_key_iv: "efcad6dc00f59b3feb5f8a4b532808f7"
server_key_version: "encryptionkey@0"
I20260807 05:08:11.922835 18216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:11.933106 18216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:11.935626 18216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:11.935817 18216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:11.943077 18216 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:35931
I20260807 05:08:11.943359 18287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:35931 every 8 connection(s)
I20260807 05:08:11.944094 18216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:11.944993 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 18216
I20260807 05:08:11.945446 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:37695
--local_ip_for_outbound_sockets=127.1.17.193
--tserver_master_addrs=127.1.17.254:35931
--webserver_port=42551
--webserver_interface=127.1.17.193
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:08:11.948388 18288 sys_catalog.cc:263] Verifying existing consensus state
I20260807 05:08:11.950062 18288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Bootstrap starting.
I20260807 05:08:11.968104 18288 log.cc:824] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:11.979303 18288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:08:11.979701 18288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Bootstrap complete.
I20260807 05:08:11.983786 18288 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:08:11.984196 18288 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10bc2c1d41748deb8e794750be53354, State: Initialized, Role: FOLLOWER
I20260807 05:08:11.984707 18288 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:08:11.984896 18288 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:11.984989 18288 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:11.985087 18288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 2 FOLLOWER]: Advancing to term 3
I20260807 05:08:11.986884 18288 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:08:11.987212 18288 leader_election.cc:302] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10bc2c1d41748deb8e794750be53354; no voters:
I20260807 05:08:11.987522 18288 leader_election.cc:288] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 05:08:11.987660 18292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 05:08:11.987972 18292 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [term 3 LEADER]: Becoming Leader. State: Replica: c10bc2c1d41748deb8e794750be53354, State: Running, Role: LEADER
I20260807 05:08:11.988288 18292 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } }
I20260807 05:08:11.988747 18288 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:11.989306 18294 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10bc2c1d41748deb8e794750be53354. Latest consensus state: current_term: 3 leader_uuid: "c10bc2c1d41748deb8e794750be53354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } } }
I20260807 05:08:11.989475 18294 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:11.989718 18293 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c10bc2c1d41748deb8e794750be53354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10bc2c1d41748deb8e794750be53354" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 35931 } } }
I20260807 05:08:11.989830 18293 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:11.991353 18297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:11.996481 18297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:11.997184 18297 catalog_manager.cc:1259] Loaded cluster ID: 9e1c73cd8da244d2b960ccde28abe0dd
I20260807 05:08:11.997248 18297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:11.999922 18297 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:12.000844 18297 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Loaded TSK: 0
I20260807 05:08:12.000927 18297 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c10bc2c1d41748deb8e794750be53354: Loaded TSK: 1
F20260807 05:08:12.002316 18297 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fed6e39d9ad google::LogMessage::Fail() at ??:0
@ 0x7fed6e3a088a google::LogMessage::SendToLog() at ??:0
@ 0x7fed6e39d4de google::LogMessage::Flush() at ??:0
@ 0x7fed6e39ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fed7106118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fed71059f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fed78d027bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fed78cdc99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fed78cdd178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fed78d1edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fed78d3d82b std::function<>::operator()() at ??:0
@ 0x7fed78cdcb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fed78cddf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fed78cda579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fed78d1e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fed705ef3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fed705efc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fed705f13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fed705e0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fed71c246db start_thread at ??:0
@ 0x7fed6ce9b61f clone at ??:0
W20260807 05:08:12.094573 18290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:12.094885 18290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:12.094965 18290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:12.101423 18290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:12.101490 18290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:12.101588 18290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:08:12.109483 18290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:37695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=42551
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:12.110885 18290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:12.112282 18290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:12.123589 18339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:12.123585 18337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:12.123764 18290 server_base.cc:1034] running on GCE node
W20260807 05:08:12.123585 18336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:12.124444 18290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:12.125233 18290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:12.126435 18290 hybrid_clock.cc:648] HybridClock initialized: now 1786079292126402 us; error 42 us; skew 500 ppm
Aug 07 05:08:12 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:12.129652 18290 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:08:12.130457 18290 webserver.cc:460] Webserver started at http://127.1.17.193:42551/ using document root <none> and password file <none>
I20260807 05:08:12.131052 18290 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:12.131114 18290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:12.135015 18290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:08:12.137234 18346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:12.138525 18290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:12.138651 18290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "68369020f360442789263055a3e2d552"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "dc23ed44d48b1257987c9dbcd2b44814"
server_key_iv: "fc53c65160376f967b16e6a3a340f305"
server_key_version: "encryptionkey@0"
I20260807 05:08:12.139204 18290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:12.157563 18290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:12.160657 18290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:12.160892 18290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:12.161482 18290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:12.162633 18353 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 05:08:12.168047 18290 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 05:08:12.168147 18290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 05:08:12.168228 18290 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 05:08:12.169881 18290 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 05:08:12.169939 18290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 05:08:12.170202 18353 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Bootstrap starting.
I20260807 05:08:12.180961 18290 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:37695
I20260807 05:08:12.182113 18290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:08:12.183339 18464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:37695 every 8 connection(s)
I20260807 05:08:12.183754 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 18290
I20260807 05:08:12.184335 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:42299
--local_ip_for_outbound_sockets=127.1.17.194
--tserver_master_addrs=127.1.17.254:35931
--webserver_port=38861
--webserver_interface=127.1.17.194
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:08:12.192878 18353 log.cc:824] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:12.202283 18353 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 05:08:12.202710 18353 tablet_bootstrap.cc:492] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Bootstrap complete.
I20260807 05:08:12.204277 18353 ts_tablet_manager.cc:1397] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.008s
I20260807 05:08:12.207600 18353 raft_consensus.cc:348] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:08:12.208105 18353 raft_consensus.cc:729] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68369020f360442789263055a3e2d552, State: Initialized, Role: FOLLOWER
I20260807 05:08:12.208619 18353 consensus_queue.cc:260] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:08:12.209630 18353 ts_tablet_manager.cc:1428] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260807 05:08:12.246872 18467 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:08:12.184449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:33781)
0807 05:08:12.186280 (+ 1831us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.186338 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:12.187366 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:12.246133 (+ 58767us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1601,"thread_start_us":134,"threads_started":1}
W20260807 05:08:12.247488 18465 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:35931 (0 consecutive failures): Network error: Failed to ping master at 127.1.17.254:35931: Client connection negotiation failed: client connection to 127.1.17.254:35931: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 05:08:12.248279 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.248020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:40113)
0807 05:08:12.248137 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.248225 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 05:08:12.248930 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.248717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:34417)
0807 05:08:12.248831 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.248876 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 05:08:12.249126 18465 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 05:08:12.309981 18468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:12.310242 18468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:12.310313 18468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:12.314266 18468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:12.314337 18468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:12.314431 18468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:08:12.319435 18468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:42299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=38861
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:12.320441 18468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:12.321450 18468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:12.330761 18479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:12.330775 18476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:12.330775 18475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:12.331043 18468 server_base.cc:1034] running on GCE node
I20260807 05:08:12.331522 18468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:12.332075 18468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:12.333225 18468 hybrid_clock.cc:648] HybridClock initialized: now 1786079292333188 us; error 50 us; skew 500 ppm
Aug 07 05:08:12 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:12.335871 18468 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:08:12.336817 18468 webserver.cc:460] Webserver started at http://127.1.17.194:38861/ using document root <none> and password file <none>
I20260807 05:08:12.337379 18468 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:12.337474 18468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:12.341158 18468 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 05:08:12.343221 18487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:12.344491 18468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:12.344633 18468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "8be603960c4340359a30369bf3e1c8eb"
format_stamp: "Formatted at 2026-08-07 05:07:29 on dist-test-slave-1x4g"
server_key: "39a5f06c43293e428489007811887e0b"
server_key_iv: "f786ea9331f0f510ac916d019f18b73b"
server_key_version: "encryptionkey@0"
I20260807 05:08:12.345207 18468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:12.362025 18468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:12.365235 18468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:12.365429 18468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:12.365974 18468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:12.366746 18468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:12.366791 18468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:12.366863 18468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:12.366896 18468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:12.375332 18468 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:42299
I20260807 05:08:12.375396 18602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:42299 every 8 connection(s)
I20260807 05:08:12.376359 18468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:08:12.380170 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.378418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:59809)
0807 05:08:12.378888 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.379001 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":118,"threads_started":1}
W20260807 05:08:12.380645 18603 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:35931 (0 consecutive failures): Network error: Failed to ping master at 127.1.17.254:35931: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
I20260807 05:08:12.381210 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.381003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:44517)
0807 05:08:12.381115 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.381159 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 05:08:12.381767 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.381577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:60589)
0807 05:08:12.381676 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.381723 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 05:08:12.381915 18603 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 05:08:12.383139 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 18468
I20260807 05:08:12.383545 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:33751
--local_ip_for_outbound_sockets=127.1.17.195
--tserver_master_addrs=127.1.17.254:35931
--webserver_port=44717
--webserver_interface=127.1.17.195
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:12.494437 18606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:12.494745 18606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:12.494817 18606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:12.498883 18606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:12.498955 18606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:12.499042 18606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:08:12.504200 18606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:33751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=44717
--tserver_master_addrs=127.1.17.254:35931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:12.505246 18606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:12.506275 18606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:12.515887 18612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:12.515925 18606 server_base.cc:1034] running on GCE node
W20260807 05:08:12.515807 18611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:12.515807 18615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:12.516539 18606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:12.517191 18606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:12.518357 18606 hybrid_clock.cc:648] HybridClock initialized: now 1786079292518343 us; error 24 us; skew 500 ppm
Aug 07 05:08:12 dist-test-slave-1x4g krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:12.521064 18606 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:08:12.521881 18606 webserver.cc:460] Webserver started at http://127.1.17.195:44717/ using document root <none> and password file <none>
I20260807 05:08:12.522423 18606 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:12.522482 18606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:12.526180 18606 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:12.528344 18623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:12.529616 18606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:12.529742 18606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "06a14d884ecc4421a15e3f8a3120becc"
format_stamp: "Formatted at 2026-08-07 05:07:30 on dist-test-slave-1x4g"
server_key: "caf4ffd83a5075d78eb1c9a3f86db1f1"
server_key_iv: "3b7d3f4b2964a89d4d0188862ead3186"
server_key_version: "encryptionkey@0"
I20260807 05:08:12.530268 18606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:12.558842 18606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:12.561721 18606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:12.561913 18606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:12.562505 18606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:12.563378 18606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:12.563438 18606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:12.563498 18606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:12.563531 18606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:12.572669 18606 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:33751
I20260807 05:08:12.572700 18740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:33751 every 8 connection(s)
I20260807 05:08:12.573621 18606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:08:12.577503 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.575764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:42835)
0807 05:08:12.576195 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.576295 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":115,"threads_started":1}
W20260807 05:08:12.578025 18741 heartbeater.cc:644] Failed to heartbeat to 127.1.17.254:35931 (0 consecutive failures): Network error: Failed to ping master at 127.1.17.254:35931: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
I20260807 05:08:12.578572 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.578388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:38581)
0807 05:08:12.578478 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.578522 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 05:08:12.579118 18743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:12.578944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:42225)
0807 05:08:12.579035 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:12.579072 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 05:08:12.579340 18741 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 05:08:12.580369 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 18606
I20260807 05:08:13.250154 18745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:13.249682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:35473)
0807 05:08:13.249984 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:13.250039 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
I20260807 05:08:13.382886 18746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:13.382472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:48029)
0807 05:08:13.382728 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:13.382786 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260807 05:08:13.580214 18747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:13.579818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:53003)
0807 05:08:13.580060 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:13.580111 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":68,"threads_started":1}
I20260807 05:08:13.614761 18748 raft_consensus.cc:482] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:13.615020 18748 raft_consensus.cc:504] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:08:13.616504 18748 leader_election.cc:288] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299), 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751)
Aug 07 05:08:13 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.195@KRBTEST.COM
Aug 07 05:08:13 dist-test-slave-1x4g krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.194@KRBTEST.COM
I20260807 05:08:13.631299 18752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:13.617100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:39861 (local address 127.1.17.194:42299)
0807 05:08:13.617498 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:13.617523 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:13.617568 (+ 45us) server_negotiation.cc:407] Connection header received
0807 05:08:13.618504 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:13.618513 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:13.618947 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:13.619254 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:13.620519 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.621765 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.622941 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.623415 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.623615 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:13.626671 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:13.626677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:13.626692 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:13.626726 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:13.628889 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:13.629382 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:13.629387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:13.629390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:13.629510 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:13.629859 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:13.629864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:13.629866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:13.630243 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:13.630383 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:13.630885 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 05:08:13.631132 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":79,"threads_started":1}
I20260807 05:08:13.631285 18745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:13.616678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:42299 (local address 127.1.17.193:39861)
0807 05:08:13.616924 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:13.616938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:13.617065 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:13.619302 (+ 2237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:13.619307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:13.619887 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:13.620346 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:13.620361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.621909 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.621912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:13.622823 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:13.622832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.623555 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.623558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:13.623768 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:13.624538 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:13.624554 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:13.626478 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:13.629028 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:13.629033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:13.629044 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:13.629283 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:13.629613 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:13.629616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:13.629618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:13.629764 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:13.630361 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:13.630367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:13.630592 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 05:08:13.630765 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 05:08:13.631079 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":82,"spinlock_wait_cycles":5760}
I20260807 05:08:13.631381 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:13.616756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33751 (local address 127.1.17.193:52881)
0807 05:08:13.617306 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:13.617319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:13.617415 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:13.619303 (+ 1888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:13.619308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:13.619887 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:13.620347 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:13.620362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.621928 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.621932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:13.622822 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:13.622831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.623554 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.623557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:13.623769 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:13.624589 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:13.624601 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:13.626132 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:13.629376 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:13.629377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:13.629379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:13.629539 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:13.629867 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:13.629872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:13.629874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:13.629955 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:13.630508 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:13.630511 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:13.630623 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 05:08:13.630762 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 05:08:13.631168 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":243,"spinlock_wait_cycles":27520,"thread_start_us":71,"threads_started":1}
I20260807 05:08:13.632048 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:13.616969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:52881 (local address 127.1.17.195:33751)
0807 05:08:13.617338 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:13.617368 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:13.617415 (+ 47us) server_negotiation.cc:407] Connection header received
0807 05:08:13.618527 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:13.618536 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:13.618948 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:13.619254 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:13.620516 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.621791 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.622950 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:13.623423 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:13.623615 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:13.626688 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:13.626693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:13.626703 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:13.626731 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:13.628889 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:13.629643 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:13.629649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:13.629654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:13.629770 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:13.630041 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:13.630046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:13.630047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:13.630400 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:13.630539 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:13.630889 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 05:08:13.631167 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":107,"threads_started":1}
W20260807 05:08:13.633886 18350 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
W20260807 05:08:13.634056 18348 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
I20260807 05:08:13.634150 18348 leader_election.cc:302] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68369020f360442789263055a3e2d552; no voters: 06a14d884ecc4421a15e3f8a3120becc, 8be603960c4340359a30369bf3e1c8eb
I20260807 05:08:13.634431 18748 raft_consensus.cc:2726] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:08:14.251324 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:14.250948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:36873)
0807 05:08:14.251176 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:14.251228 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":73,"threads_started":1}
I20260807 05:08:14.383903 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:14.383548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:52435)
0807 05:08:14.383771 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:14.383821 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260807 05:08:14.581319 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:14.580961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:59961)
0807 05:08:14.581196 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:14.581242 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":95,"threads_started":1}
I20260807 05:08:15.252532 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:15.252091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:52921)
0807 05:08:15.252367 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:15.252434 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260807 05:08:15.385058 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:15.384582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:36271)
0807 05:08:15.384885 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:15.384936 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260807 05:08:15.460335 18758 raft_consensus.cc:482] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:15.460471 18758 raft_consensus.cc:504] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:08:15.461047 18758 leader_election.cc:288] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299), 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751)
W20260807 05:08:15.461802 18350 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
W20260807 05:08:15.461944 18348 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
I20260807 05:08:15.462037 18348 leader_election.cc:302] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68369020f360442789263055a3e2d552; no voters: 06a14d884ecc4421a15e3f8a3120becc, 8be603960c4340359a30369bf3e1c8eb
I20260807 05:08:15.462198 18758 raft_consensus.cc:2726] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:08:15.582368 18759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:15.581967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:54499)
0807 05:08:15.582223 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:15.582273 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260807 05:08:16.253757 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:16.253373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:45213)
0807 05:08:16.253603 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:16.253666 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260807 05:08:16.386106 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:16.385730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:43085)
0807 05:08:16.385970 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:16.386023 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":73,"threads_started":1}
I20260807 05:08:16.583588 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:16.583197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:50109)
0807 05:08:16.583445 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:16.583499 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260807 05:08:17.255002 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:17.254591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:55885)
0807 05:08:17.254853 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:17.254908 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":70,"threads_started":1}
I20260807 05:08:17.360389 18764 raft_consensus.cc:482] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:17.360553 18764 raft_consensus.cc:504] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:08:17.361042 18764 leader_election.cc:288] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299), 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751)
W20260807 05:08:17.361828 18350 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
W20260807 05:08:17.362027 18348 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
I20260807 05:08:17.362102 18348 leader_election.cc:302] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68369020f360442789263055a3e2d552; no voters: 06a14d884ecc4421a15e3f8a3120becc, 8be603960c4340359a30369bf3e1c8eb
I20260807 05:08:17.362282 18764 raft_consensus.cc:2726] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:08:17.387169 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:17.386807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:37271)
0807 05:08:17.387036 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:17.387088 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
I20260807 05:08:17.584671 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:17.584290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:46945)
0807 05:08:17.584542 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:17.584591 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
I20260807 05:08:18.256352 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:18.255874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:57835)
0807 05:08:18.256174 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:18.256241 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":75,"threads_started":1}
I20260807 05:08:18.388453 18768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:18.388039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:37939)
0807 05:08:18.388297 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:18.388360 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260807 05:08:18.586023 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:18.585529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:57689)
0807 05:08:18.585838 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:18.585902 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":98,"threads_started":1}
I20260807 05:08:19.257630 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:19.257206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:60057)
0807 05:08:19.257478 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:19.257531 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260807 05:08:19.389600 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:19.389156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:40231)
0807 05:08:19.389458 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:19.389511 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":85,"threads_started":1}
I20260807 05:08:19.587383 18772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:19.586875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:37077)
0807 05:08:19.587239 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:19.587293 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":69,"threads_started":1}
I20260807 05:08:20.258695 18773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:20.258310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:51855)
0807 05:08:20.258551 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:20.258601 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":71,"threads_started":1}
I20260807 05:08:20.390688 18774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:20.390296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:36401)
0807 05:08:20.390542 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:20.390601 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260807 05:08:20.588537 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:20.588093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:47251)
0807 05:08:20.588374 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:20.588442 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260807 05:08:21.259876 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:21.259473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:47631)
0807 05:08:21.259722 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:21.259775 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260807 05:08:21.391839 18778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:21.391400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:51581)
0807 05:08:21.391666 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:21.391723 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
I20260807 05:08:21.589630 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:21.589231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:46637)
0807 05:08:21.589491 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:21.589543 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":93,"threads_started":1}
I20260807 05:08:22.195221 18780 raft_consensus.cc:482] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:22.195358 18780 raft_consensus.cc:504] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:08:22.195825 18780 leader_election.cc:288] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299), 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751)
W20260807 05:08:22.196717 18350 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
W20260807 05:08:22.196887 18348 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
I20260807 05:08:22.196971 18348 leader_election.cc:302] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68369020f360442789263055a3e2d552; no voters: 06a14d884ecc4421a15e3f8a3120becc, 8be603960c4340359a30369bf3e1c8eb
I20260807 05:08:22.197129 18780 raft_consensus.cc:2726] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:08:22.261197 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:22.260739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:45507)
0807 05:08:22.261041 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:22.261099 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":132,"threads_started":1}
I20260807 05:08:22.393003 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:22.392590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:48613)
0807 05:08:22.392857 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:22.392914 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":68,"threads_started":1}
I20260807 05:08:22.590721 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:22.590332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:58411)
0807 05:08:22.590580 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:22.590632 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260807 05:08:23.262332 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:23.261917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:43137)
0807 05:08:23.262183 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:23.262236 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":77,"threads_started":1}
I20260807 05:08:23.394109 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:23.393724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:39469)
0807 05:08:23.393966 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:23.394017 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260807 05:08:23.591804 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:23.591377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:58981)
0807 05:08:23.591635 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:23.591703 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260807 05:08:24.263494 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:24.263074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:46399)
0807 05:08:24.263335 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:24.263388 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":96,"threads_started":1}
I20260807 05:08:24.395259 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:24.394830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:59947)
0807 05:08:24.395112 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:24.395165 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":123,"threads_started":1}
I20260807 05:08:24.592949 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:24.592486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:34627)
0807 05:08:24.592741 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:24.592850 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260807 05:08:25.264680 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:25.264262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:53731)
0807 05:08:25.264518 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:25.264582 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
I20260807 05:08:25.396428 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:25.396020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:40555)
0807 05:08:25.396281 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:25.396334 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260807 05:08:25.594208 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:25.593743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:51803)
0807 05:08:25.594033 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:25.594091 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260807 05:08:26.265848 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:26.265442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:41111)
0807 05:08:26.265697 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:26.265751 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
I20260807 05:08:26.397619 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:26.397219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:42051)
0807 05:08:26.397470 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:26.397524 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260807 05:08:26.595378 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:26.594999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:52231)
0807 05:08:26.595234 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:26.595286 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
I20260807 05:08:27.154198 18796 raft_consensus.cc:482] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:27.154356 18796 raft_consensus.cc:504] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8be603960c4340359a30369bf3e1c8eb" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 42299 } } peers { permanent_uuid: "06a14d884ecc4421a15e3f8a3120becc" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33751 } } peers { permanent_uuid: "68369020f360442789263055a3e2d552" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 37695 } }
I20260807 05:08:27.154814 18796 leader_election.cc:288] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299), 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751)
W20260807 05:08:27.155548 18350 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8be603960c4340359a30369bf3e1c8eb (127.1.17.194:42299): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
W20260807 05:08:27.155671 18348 leader_election.cc:341] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06a14d884ecc4421a15e3f8a3120becc (127.1.17.195:33751): Not found: Tablet not found: 8982fb7467244861a8c8c2697aa95e85
I20260807 05:08:27.155766 18348 leader_election.cc:302] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68369020f360442789263055a3e2d552; no voters: 06a14d884ecc4421a15e3f8a3120becc, 8be603960c4340359a30369bf3e1c8eb
I20260807 05:08:27.155932 18796 raft_consensus.cc:2726] T 8982fb7467244861a8c8c2697aa95e85 P 68369020f360442789263055a3e2d552 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 05:08:27.267298 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:27.266883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.193:56567)
0807 05:08:27.267144 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:27.267198 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
I20260807 05:08:27.398873 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:27.398455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.194:44981)
0807 05:08:27.398721 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:27.398779 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":89,"threads_started":1}
I20260807 05:08:27.585947 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 18290
I20260807 05:08:27.594376 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 18468
I20260807 05:08:27.596865 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:27.596362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:35931 (local address 127.1.17.195:47611)
0807 05:08:27.596627 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:27.596697 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.254:35931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260807 05:08:27.601461 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 18606
2026-08-07T05:08:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59556 ms)
[----------] 30 tests from SecurityITest (239521 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:27 dist-test-slave-1x4g krb5kdc[18802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:27 dist-test-slave-1x4g krb5kdc[18802](info): set up 2 sockets
Aug 07 05:08:27 dist-test-slave-1x4g krb5kdc[18802](info): commencing operation
krb5kdc: starting...
W20260807 05:08:28.653770 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:28 dist-test-slave-1x4g krb5kdc[18802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:08:28.781684 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39073
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:39073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:28.893451 18815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:28.893728 18815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:28.893822 18815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:28.897708 18815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:28.897782 18815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:28.897820 18815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:28.897876 18815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:28.897938 18815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:28.903015 18815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:39073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:28.904033 18815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:28.905092 18815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:28.913697 18824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:28.913786 18822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:28.913736 18815 server_base.cc:1034] running on GCE node
W20260807 05:08:28.913743 18821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:28.914531 18815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:28.915512 18815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:28.916680 18815 hybrid_clock.cc:648] HybridClock initialized: now 1786079308916660 us; error 31 us; skew 500 ppm
Aug 07 05:08:28 dist-test-slave-1x4g krb5kdc[18802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079308, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:28.924427 18815 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:08:28.925400 18815 webserver.cc:460] Webserver started at http://127.1.17.254:39505/ using document root <none> and password file <none>
I20260807 05:08:28.925910 18815 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:28.925967 18815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:28.926138 18815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:28.928145 18815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "fcb39eb8c9a34471a3e3834999ca302c"
format_stamp: "Formatted at 2026-08-07 05:08:28 on dist-test-slave-1x4g"
server_key: "c20cd4b82ee86a68879622785f87be06"
server_key_iv: "3b63ed78670c0ce8514d933c81c7568c"
server_key_version: "encryptionkey@0"
I20260807 05:08:28.928676 18815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "fcb39eb8c9a34471a3e3834999ca302c"
format_stamp: "Formatted at 2026-08-07 05:08:28 on dist-test-slave-1x4g"
server_key: "c20cd4b82ee86a68879622785f87be06"
server_key_iv: "3b63ed78670c0ce8514d933c81c7568c"
server_key_version: "encryptionkey@0"
I20260807 05:08:28.932307 18815 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:08:28.934617 18831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:28.935837 18815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:28.936009 18815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "fcb39eb8c9a34471a3e3834999ca302c"
format_stamp: "Formatted at 2026-08-07 05:08:28 on dist-test-slave-1x4g"
server_key: "c20cd4b82ee86a68879622785f87be06"
server_key_iv: "3b63ed78670c0ce8514d933c81c7568c"
server_key_version: "encryptionkey@0"
I20260807 05:08:28.936120 18815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:28.957427 18815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:28.960054 18815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:28.960243 18815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:28.967402 18815 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:39073
I20260807 05:08:28.967432 18883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:39073 every 8 connection(s)
I20260807 05:08:28.968330 18815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:28.971058 18884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:28.976269 18884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c: Bootstrap starting.
I20260807 05:08:28.977584 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 18815
I20260807 05:08:28.977687 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:28.978003 1095 external_mini_cluster.cc:1428] Setting key e826fe9204c24042adbc085275ad942c
I20260807 05:08:28.979154 18884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:28.980075 18884 log.cc:824] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:28.982328 18884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c: No bootstrap required, opened a new log
I20260807 05:08:28.986155 18884 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb39eb8c9a34471a3e3834999ca302c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39073 } }
I20260807 05:08:28.986429 18884 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:28.986490 18884 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb39eb8c9a34471a3e3834999ca302c, State: Initialized, Role: FOLLOWER
I20260807 05:08:28.986909 18884 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb39eb8c9a34471a3e3834999ca302c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39073 } }
I20260807 05:08:28.987038 18884 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:28.987129 18884 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:28.987254 18884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:28.988103 18884 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb39eb8c9a34471a3e3834999ca302c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39073 } }
Aug 07 05:08:28 dist-test-slave-1x4g krb5kdc[18802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:28.988449 18884 leader_election.cc:302] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb39eb8c9a34471a3e3834999ca302c; no voters:
I20260807 05:08:28.988781 18884 leader_election.cc:288] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:28.988874 18889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:28.989105 18889 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [term 1 LEADER]: Becoming Leader. State: Replica: fcb39eb8c9a34471a3e3834999ca302c, State: Running, Role: LEADER
I20260807 05:08:28.989466 18889 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb39eb8c9a34471a3e3834999ca302c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39073 } }
I20260807 05:08:28.990114 18884 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:28.991168 18890 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcb39eb8c9a34471a3e3834999ca302c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb39eb8c9a34471a3e3834999ca302c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39073 } } }
I20260807 05:08:28.991289 18890 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:28.991703 18891 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb39eb8c9a34471a3e3834999ca302c. Latest consensus state: current_term: 1 leader_uuid: "fcb39eb8c9a34471a3e3834999ca302c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb39eb8c9a34471a3e3834999ca302c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39073 } } }
I20260807 05:08:28.991897 18891 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:28.994743 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:28.979114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40048 (local address 127.1.17.254:39073)
0807 05:08:28.979598 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:28.979618 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:28.979659 (+ 41us) server_negotiation.cc:407] Connection header received
0807 05:08:28.980556 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:28.980576 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:28.980981 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:28.981341 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:28.981993 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:28.983395 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:28.984273 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:28.985851 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:28.986001 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:28.988729 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:28.988734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:28.988747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:28.988818 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:28.991083 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:28.991638 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:28.991643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:28.991649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:28.991802 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:28.992133 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:28.992136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:28.992137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:28.993074 (+ 937us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:28.993259 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:28.993545 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 05:08:28.993830 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":138,"threads_started":1}
W20260807 05:08:28.995522 18905 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:28.995595 18905 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:28.995671 18898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:28.996481 18898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:29.002108 18898 catalog_manager.cc:1347] Generated new cluster ID: 14ca39c02a6943b1be99aba563a8bdec
I20260807 05:08:29.002174 18898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:29.014412 18898 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:29.015599 18898 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:29.021258 18898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c: Generated new TSK 0
I20260807 05:08:29.021885 18898 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:08:29.059458 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39073
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:29.168365 18910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:29.168579 18910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:29.168617 18910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:29.172375 18910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:29.172418 18910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:29.172469 18910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:08:29.177259 18910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:29.178139 18910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:29.179013 18910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:29.188004 18918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:29.188032 18920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:29.188158 18910 server_base.cc:1034] running on GCE node
W20260807 05:08:29.188032 18916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:29.188596 18910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:29.189265 18910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:29.190461 18910 hybrid_clock.cc:648] HybridClock initialized: now 1786079309190453 us; error 36 us; skew 500 ppm
Aug 07 05:08:29 dist-test-slave-1x4g krb5kdc[18802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079309, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:29.193452 18910 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:08:29.194346 18910 webserver.cc:460] Webserver started at http://127.1.17.193:43157/ using document root <none> and password file <none>
I20260807 05:08:29.194952 18910 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:29.195050 18910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:29.195276 18910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:29.197261 18910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "874ab346d8434a36942370f8e9f8f3ed"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "02a89521166b1ea3b1cc765baf61bdf4"
server_key_iv: "237bdeeff2331e35eabd1f9e35ab66a1"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.197832 18910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "874ab346d8434a36942370f8e9f8f3ed"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "02a89521166b1ea3b1cc765baf61bdf4"
server_key_iv: "237bdeeff2331e35eabd1f9e35ab66a1"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.201462 18910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:08:29.203776 18927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.204967 18910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:29.205083 18910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "874ab346d8434a36942370f8e9f8f3ed"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "02a89521166b1ea3b1cc765baf61bdf4"
server_key_iv: "237bdeeff2331e35eabd1f9e35ab66a1"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.205192 18910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:29.222551 18910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:29.225214 18910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:29.225412 18910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:29.225986 18910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:29.226771 18910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:29.226850 18910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.226929 18910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:29.226981 18910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.236006 18910 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:35225
I20260807 05:08:29.236028 19044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:35225 every 8 connection(s)
I20260807 05:08:29.236954 18910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
Aug 07 05:08:29 dist-test-slave-1x4g krb5kdc[18802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079309, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:29.245795 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 18910
I20260807 05:08:29.245926 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:08:29.246214 1095 external_mini_cluster.cc:1428] Setting key 2882bf0b3c4134899be65c71854b97de
I20260807 05:08:29.249800 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.238639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:38243 (local address 127.1.17.254:39073)
0807 05:08:29.238794 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:29.238798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:29.239669 (+ 871us) server_negotiation.cc:407] Connection header received
0807 05:08:29.240577 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:29.240580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:29.240628 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:29.240700 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:29.241847 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.242378 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.243450 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.243675 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.243751 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.246090 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:29.246093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:29.246095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:29.246117 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:29.247717 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.248215 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.248218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.248220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.248300 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.248630 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.248634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.248636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.248884 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:29.249002 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:29.249503 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 05:08:29.249634 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 05:08:29.250514 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.239050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39073 (local address 127.1.17.193:38243)
0807 05:08:29.239531 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:29.239570 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:29.240354 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:29.240903 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:29.240910 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:29.241255 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:29.241701 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.241714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.242491 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.242493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.243347 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.243355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.243798 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.243801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.243934 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.244444 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:29.244459 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:29.245893 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:29.247842 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:29.247847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:29.247860 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:29.248114 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:29.248397 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:29.248401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:29.248403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:29.248538 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:29.249006 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:29.249015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:29.249248 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 05:08:29.249436 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 05:08:29.249734 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":150,"threads_started":1}
I20260807 05:08:29.251821 19045 heartbeater.cc:347] Connected to a master server at 127.1.17.254:39073
I20260807 05:08:29.252053 19045 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:29.252496 19045 heartbeater.cc:511] Master 127.1.17.254:39073 requested a full tablet report, sending...
I20260807 05:08:29.254312 18848 ts_manager.cc:194] Registered new tserver with Master: 874ab346d8434a36942370f8e9f8f3ed (127.1.17.193:35225)
I20260807 05:08:29.255964 18848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:38243
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:08:29.276271 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39073
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:29.386243 19050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:29.386514 19050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:29.386591 19050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:29.390851 19050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:29.390936 19050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:29.391054 19050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:08:29.396293 19050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:29.397305 19050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:29.398307 19050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:29.407748 19055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:29.407755 19056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:29.407835 19050 server_base.cc:1034] running on GCE node
W20260807 05:08:29.407763 19059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:29.408412 19050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:29.409096 19050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:29.410276 19050 hybrid_clock.cc:648] HybridClock initialized: now 1786079309410230 us; error 57 us; skew 500 ppm
Aug 07 05:08:29 dist-test-slave-1x4g krb5kdc[18802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079309, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:29.413048 19050 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:08:29.413861 19050 webserver.cc:460] Webserver started at http://127.1.17.194:43799/ using document root <none> and password file <none>
I20260807 05:08:29.414458 19050 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:29.414546 19050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:29.414775 19050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:29.416767 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "f6dfc37fb12844a48f11c3de4e60c19f"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "845615f940dd9612a8c3e7eb1e79e81b"
server_key_iv: "496e79a7f70934b64c7b58ec06c142f4"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.417385 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "f6dfc37fb12844a48f11c3de4e60c19f"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "845615f940dd9612a8c3e7eb1e79e81b"
server_key_iv: "496e79a7f70934b64c7b58ec06c142f4"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.420949 19050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:29.423229 19066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.424409 19050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:29.424559 19050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "f6dfc37fb12844a48f11c3de4e60c19f"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "845615f940dd9612a8c3e7eb1e79e81b"
server_key_iv: "496e79a7f70934b64c7b58ec06c142f4"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.424662 19050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:29.452541 19050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:29.455737 19050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:29.455940 19050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:29.456526 19050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:29.457435 19050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:29.457506 19050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.457579 19050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:29.457621 19050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.466768 19050 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:35427
I20260807 05:08:29.466794 19182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:35427 every 8 connection(s)
I20260807 05:08:29.467608 19050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:08:29.472533 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 19050
I20260807 05:08:29.472669 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:08:29.473007 1095 external_mini_cluster.cc:1428] Setting key ae7c3fd36af7bc3882e9cdc13453c231
Aug 07 05:08:29 dist-test-slave-1x4g krb5kdc[18802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079309, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:29.481551 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.469327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:52199 (local address 127.1.17.254:39073)
0807 05:08:29.469450 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:29.469454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:29.470184 (+ 730us) server_negotiation.cc:407] Connection header received
0807 05:08:29.471093 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:29.471096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:29.471157 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:29.471262 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:29.472406 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.473139 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.474424 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.474660 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.474769 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.477448 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:29.477453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:29.477457 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:29.477490 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:29.479197 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.479697 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.479703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.479708 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.479818 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.480147 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.480152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.480156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.480422 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:29.480542 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:29.481171 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 05:08:29.481317 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 05:08:29.482229 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.469626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39073 (local address 127.1.17.194:52199)
0807 05:08:29.470043 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:29.470083 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:29.470884 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:29.471445 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:29.471452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:29.471789 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:29.472245 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.472258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.473263 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.473272 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.474291 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.474303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.474812 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.474816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.474984 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.475548 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:29.475563 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:29.477273 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:29.479320 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:29.479331 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:29.479342 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:29.479592 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:29.479921 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:29.479923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:29.479924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:29.480048 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:29.480566 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:29.480571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:29.480868 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 05:08:29.481090 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 05:08:29.481359 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":100,"threads_started":1}
I20260807 05:08:29.483435 19183 heartbeater.cc:347] Connected to a master server at 127.1.17.254:39073
I20260807 05:08:29.483675 19183 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:29.484158 19183 heartbeater.cc:511] Master 127.1.17.254:39073 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:08:29.485329 18848 ts_manager.cc:194] Registered new tserver with Master: f6dfc37fb12844a48f11c3de4e60c19f (127.1.17.194:35427)
I20260807 05:08:29.485966 18848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:52199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:08:29.503154 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39073
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:29.617470 19188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:29.617727 19188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:29.617810 19188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:29.621788 19188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:29.621851 19188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:29.621973 19188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:08:29.627115 19188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:39073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:29.628166 19188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:29.629226 19188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:29.638990 19197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:29.639150 19188 server_base.cc:1034] running on GCE node
W20260807 05:08:29.639045 19195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:29.639178 19193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:29.639789 19188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:29.640479 19188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:29.641665 19188 hybrid_clock.cc:648] HybridClock initialized: now 1786079309641653 us; error 30 us; skew 500 ppm
Aug 07 05:08:29 dist-test-slave-1x4g krb5kdc[18802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079309, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:29.644479 19188 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:08:29.645360 19188 webserver.cc:460] Webserver started at http://127.1.17.195:38999/ using document root <none> and password file <none>
I20260807 05:08:29.645870 19188 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:29.645928 19188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:29.646113 19188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:29.648100 19188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "42af7a9808904218be2914ec3e665c2d"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "59c227319c7d8fd8577e2cab1a698c41"
server_key_iv: "f14db103dbfedc4ee91e4747a3305004"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.648645 19188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "42af7a9808904218be2914ec3e665c2d"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "59c227319c7d8fd8577e2cab1a698c41"
server_key_iv: "f14db103dbfedc4ee91e4747a3305004"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.652281 19188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:08:29.654738 19204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.655932 19188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:29.656052 19188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "42af7a9808904218be2914ec3e665c2d"
format_stamp: "Formatted at 2026-08-07 05:08:29 on dist-test-slave-1x4g"
server_key: "59c227319c7d8fd8577e2cab1a698c41"
server_key_iv: "f14db103dbfedc4ee91e4747a3305004"
server_key_version: "encryptionkey@0"
I20260807 05:08:29.656170 19188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:29.671497 19188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:29.675711 19188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:29.675904 19188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:29.676483 19188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:29.677399 19188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:29.677448 19188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.677525 19188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:29.677563 19188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:29.686354 19188 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:33687
I20260807 05:08:29.686384 19322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:33687 every 8 connection(s)
I20260807 05:08:29.687222 19188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:08:29.689301 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 19188
I20260807 05:08:29.689430 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:08:29.689875 1095 external_mini_cluster.cc:1428] Setting key 73e80d1bb657a5f27d5406813043a66b
Aug 07 05:08:29 dist-test-slave-1x4g krb5kdc[18802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079309, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:29.700482 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.688937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:53099 (local address 127.1.17.254:39073)
0807 05:08:29.689081 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:29.689085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:29.690109 (+ 1024us) server_negotiation.cc:407] Connection header received
0807 05:08:29.691380 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:29.691383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:29.691437 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:29.691567 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:29.692910 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.693432 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.694548 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.694770 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.694876 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.697266 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:29.697269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:29.697271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:29.697293 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:29.698515 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.699010 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.699018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.699019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.699090 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.699437 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.699440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.699441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.699619 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:29.699726 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:29.700194 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 05:08:29.700306 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:08:29.701304 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.689394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39073 (local address 127.1.17.195:53099)
0807 05:08:29.689944 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:29.690002 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:29.691148 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:29.691769 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:29.691776 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:29.692230 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:29.692747 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.692760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.693565 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.693568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.694426 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.694435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.694914 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.694918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.695097 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.695584 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:29.695599 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:29.697094 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:29.698633 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:29.698638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:29.698652 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:29.698895 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:29.699181 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:29.699185 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:29.699187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:29.699335 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:29.699725 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:29.699732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:29.699966 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 05:08:29.700136 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 05:08:29.700424 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":167,"threads_started":1}
I20260807 05:08:29.702378 19323 heartbeater.cc:347] Connected to a master server at 127.1.17.254:39073
I20260807 05:08:29.702653 19323 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:29.703202 19323 heartbeater.cc:511] Master 127.1.17.254:39073 requested a full tablet report, sending...
I20260807 05:08:29.704298 18848 ts_manager.cc:194] Registered new tserver with Master: 42af7a9808904218be2914ec3e665c2d (127.1.17.195:33687)
I20260807 05:08:29.704897 18848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:53099
I20260807 05:08:29.715344 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:08:29.724757 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.717399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.1.17.254:39073)
0807 05:08:29.717568 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:29.717573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:29.717711 (+ 138us) server_negotiation.cc:407] Connection header received
0807 05:08:29.717863 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:29.717868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:29.717911 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:29.717998 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:29.718754 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.719255 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.720135 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.720357 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.720459 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.721879 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:29.721883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:29.721885 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:29.721908 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:29.723049 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.723493 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.723497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.723498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.723578 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:29.723854 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:29.723857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:29.723859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:29.724074 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:29.724213 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:29.724450 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 05:08:29.724570 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 05:08:29.727422 18848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:08:29.730760 18848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:08:29.744473 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.739040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.1.17.195:33687)
0807 05:08:29.739419 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:29.739431 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:29.739458 (+ 27us) server_negotiation.cc:407] Connection header received
0807 05:08:29.739546 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:29.739551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:29.739767 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:29.739926 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:29.740270 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.741283 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.742973 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.743794 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.743938 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.744117 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:29.744249 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 05:08:29.744335 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":179,"threads_started":1}
I20260807 05:08:29.744493 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.738878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33687 (local address 127.0.0.1:34094)
0807 05:08:29.739237 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:29.739290 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:29.739412 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:29.739967 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:29.739974 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:29.739997 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:29.740152 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.740160 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.741412 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.741416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.742822 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.742832 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.743938 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.743942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.744067 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.744119 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 05:08:29.744221 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 05:08:29.744320 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260807 05:08:29.745415 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.740072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:35427 (local address 127.0.0.1:39970)
0807 05:08:29.740310 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:29.740323 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:29.740408 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:29.741474 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:29.741476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:29.741494 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:29.741581 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.741586 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.742687 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.742694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.743842 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.743854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.745025 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.745029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.745115 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.745137 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 05:08:29.745226 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 05:08:29.745294 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":109,"threads_started":1}
I20260807 05:08:29.745484 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.740650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.1.17.194:35427)
0807 05:08:29.741104 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:29.741116 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:29.741133 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:08:29.741196 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:29.741200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:29.741346 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:29.741504 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:29.741693 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.742548 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.744002 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.744853 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.745016 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.745154 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:29.745253 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 05:08:29.745337 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":134,"threads_started":1}
I20260807 05:08:29.746009 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.739595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:35225 (local address 127.0.0.1:50140)
0807 05:08:29.739820 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:29.739831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:29.739931 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:29.741680 (+ 1749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:29.741683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:29.741702 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:29.741810 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.741816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.743265 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.743268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.744333 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:29.744342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.745616 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.745620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:29.745710 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.745731 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:08:29.745811 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:08:29.745882 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
I20260807 05:08:29.746203 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:29.740527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.1.17.193:35225)
0807 05:08:29.741041 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:29.741056 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:29.741079 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:08:29.741161 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:29.741167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:29.741346 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:29.741504 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:29.741940 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.743104 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.744459 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:29.745317 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:29.745419 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:29.745548 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:29.745853 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 05:08:29.745964 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":127,"threads_started":1}
I20260807 05:08:29.747601 19253 tablet_service.cc:1467] Processing CreateTablet for tablet 978c155c77334f5bbff4c33d35c49284 (DEFAULT_TABLE table=test-table [id=90a1e62358f44d05989e532441e719ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:29.747857 19115 tablet_service.cc:1467] Processing CreateTablet for tablet 978c155c77334f5bbff4c33d35c49284 (DEFAULT_TABLE table=test-table [id=90a1e62358f44d05989e532441e719ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:29.748602 19253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 978c155c77334f5bbff4c33d35c49284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:29.748849 19115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 978c155c77334f5bbff4c33d35c49284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:29.751570 18977 tablet_service.cc:1467] Processing CreateTablet for tablet 978c155c77334f5bbff4c33d35c49284 (DEFAULT_TABLE table=test-table [id=90a1e62358f44d05989e532441e719ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:29.752580 18977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 978c155c77334f5bbff4c33d35c49284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:29.755151 19340 tablet_bootstrap.cc:492] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Bootstrap starting.
I20260807 05:08:29.756636 19341 tablet_bootstrap.cc:492] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Bootstrap starting.
I20260807 05:08:29.757676 19340 tablet_bootstrap.cc:654] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:29.758594 19340 log.cc:824] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:29.759145 19341 tablet_bootstrap.cc:654] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:29.760046 19341 log.cc:824] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:29.760569 19340 tablet_bootstrap.cc:492] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: No bootstrap required, opened a new log
I20260807 05:08:29.760825 19340 ts_tablet_manager.cc:1397] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:08:29.761703 19341 tablet_bootstrap.cc:492] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: No bootstrap required, opened a new log
I20260807 05:08:29.761843 19341 ts_tablet_manager.cc:1397] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:08:29.763767 19343 tablet_bootstrap.cc:492] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: Bootstrap starting.
I20260807 05:08:29.763698 19340 raft_consensus.cc:348] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:29.764106 19340 raft_consensus.cc:374] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:29.764207 19340 raft_consensus.cc:729] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42af7a9808904218be2914ec3e665c2d, State: Initialized, Role: FOLLOWER
I20260807 05:08:29.764663 19340 consensus_queue.cc:260] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:29.764964 19341 raft_consensus.cc:348] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:29.765287 19341 raft_consensus.cc:374] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:29.765347 19323 heartbeater.cc:503] Master 127.1.17.254:39073 was elected leader, sending a full tablet report...
I20260807 05:08:29.765383 19341 raft_consensus.cc:729] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dfc37fb12844a48f11c3de4e60c19f, State: Initialized, Role: FOLLOWER
I20260807 05:08:29.765704 19340 ts_tablet_manager.cc:1428] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:08:29.765931 19343 tablet_bootstrap.cc:654] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:29.765880 19341 consensus_queue.cc:260] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:29.766525 19183 heartbeater.cc:503] Master 127.1.17.254:39073 was elected leader, sending a full tablet report...
I20260807 05:08:29.766759 19343 log.cc:824] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:29.766810 19341 ts_tablet_manager.cc:1428] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:08:29.768680 19343 tablet_bootstrap.cc:492] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: No bootstrap required, opened a new log
I20260807 05:08:29.768909 19343 ts_tablet_manager.cc:1397] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:08:29.772639 19343 raft_consensus.cc:348] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:29.772897 19343 raft_consensus.cc:374] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:29.772941 19343 raft_consensus.cc:729] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ab346d8434a36942370f8e9f8f3ed, State: Initialized, Role: FOLLOWER
I20260807 05:08:29.773245 19343 consensus_queue.cc:260] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:29.773813 19045 heartbeater.cc:503] Master 127.1.17.254:39073 was elected leader, sending a full tablet report...
I20260807 05:08:29.774004 19343 ts_tablet_manager.cc:1428] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260807 05:08:29.938196 19324 tablet.cc:2369] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:08:29.968933 19184 tablet.cc:2369] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:08:29.988320 19046 tablet.cc:2369] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:08:30.098877 19346 raft_consensus.cc:482] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:30.099115 19346 raft_consensus.cc:504] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:30.100366 19346 leader_election.cc:288] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42af7a9808904218be2914ec3e665c2d (127.1.17.195:33687), 874ab346d8434a36942370f8e9f8f3ed (127.1.17.193:35225)
I20260807 05:08:30.102663 19348 raft_consensus.cc:482] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:30.102838 19348 raft_consensus.cc:504] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:30.103947 19348 leader_election.cc:288] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42af7a9808904218be2914ec3e665c2d (127.1.17.195:33687), f6dfc37fb12844a48f11c3de4e60c19f (127.1.17.194:35427)
I20260807 05:08:30.105173 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.101268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:43063 (local address 127.1.17.193:35225)
0807 05:08:30.101412 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.101415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.101426 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:08:30.101579 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.101582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.101632 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.101715 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:30.102024 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.102501 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.103709 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.104293 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.104406 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.104466 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.104961 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.105043 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 05:08:30.105794 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.101110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:35225 (local address 127.1.17.194:43063)
0807 05:08:30.101349 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:30.101363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:30.101456 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:30.101739 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:30.101741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:30.101756 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:30.101903 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.101918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.102641 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.102643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.103578 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.103588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.104528 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.104531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.104768 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.104826 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 05:08:30.105518 (+ 692us) client_negotiation.cc:240] Negotiation successful
0807 05:08:30.105612 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":29,"thread_start_us":64,"threads_started":1}
I20260807 05:08:30.106242 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.100834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:33123 (local address 127.1.17.195:33687)
0807 05:08:30.101003 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.101010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.101090 (+ 80us) server_negotiation.cc:407] Connection header received
0807 05:08:30.101388 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.101394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.101463 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.101537 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:30.101892 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.102479 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.104910 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.105546 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.105645 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.105721 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.106038 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.106118 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":56}
I20260807 05:08:30.106575 18998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "978c155c77334f5bbff4c33d35c49284" candidate_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ab346d8434a36942370f8e9f8f3ed" is_pre_election: true
I20260807 05:08:30.106833 18998 raft_consensus.cc:2454] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6dfc37fb12844a48f11c3de4e60c19f in term 0.
I20260807 05:08:30.106863 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.100647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33687 (local address 127.1.17.194:33123)
0807 05:08:30.101004 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:30.101020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:30.101207 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:30.101571 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:30.101574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:30.101594 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:30.101760 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.101767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.103698 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.103700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.104735 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.104749 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.105766 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.105772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.105895 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.105922 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 05:08:30.106502 (+ 580us) client_negotiation.cc:240] Negotiation successful
0807 05:08:30.106576 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":1,"thread_start_us":140,"threads_started":1}
I20260807 05:08:30.107431 19274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "978c155c77334f5bbff4c33d35c49284" candidate_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42af7a9808904218be2914ec3e665c2d" is_pre_election: true
I20260807 05:08:30.107527 19069 leader_election.cc:302] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 874ab346d8434a36942370f8e9f8f3ed, f6dfc37fb12844a48f11c3de4e60c19f; no voters:
I20260807 05:08:30.107730 19274 raft_consensus.cc:2454] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6dfc37fb12844a48f11c3de4e60c19f in term 0.
I20260807 05:08:30.107848 19346 raft_consensus.cc:2781] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:08:30.107921 19346 raft_consensus.cc:482] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:08:30.107959 19346 raft_consensus.cc:3037] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:30.109367 19346 raft_consensus.cc:504] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:30.109493 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.104230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:35427 (local address 127.1.17.193:48947)
0807 05:08:30.104770 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:30.104809 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:30.104918 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:30.105279 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:30.105285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:30.105305 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:30.105414 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.105420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.106789 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.106792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.108112 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.108122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.109054 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.109057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.109129 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.109144 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 05:08:30.109230 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 05:08:30.109298 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":98,"thread_start_us":102,"threads_started":1}
I20260807 05:08:30.109495 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.104554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:48947 (local address 127.1.17.194:35427)
0807 05:08:30.104648 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.104652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.104867 (+ 215us) server_negotiation.cc:407] Connection header received
0807 05:08:30.105042 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.105045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.105087 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.105134 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:30.105537 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.106273 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.108249 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.108864 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.108943 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.108976 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.109245 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.109309 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":27}
I20260807 05:08:30.109678 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.104122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:33687 (local address 127.1.17.193:35493)
0807 05:08:30.104724 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:30.104736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:30.104861 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:30.105405 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:30.105407 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:30.105421 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:30.105551 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.105560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.106730 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.106736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.108077 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.108091 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.109137 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.109142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.109282 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.109318 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 05:08:30.109400 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 05:08:30.109529 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":53,"thread_start_us":100,"threads_started":1}
I20260807 05:08:30.109760 19346 leader_election.cc:288] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [CANDIDATE]: Term 1 election: Requested vote from peers 42af7a9808904218be2914ec3e665c2d (127.1.17.195:33687), 874ab346d8434a36942370f8e9f8f3ed (127.1.17.193:35225)
I20260807 05:08:30.110077 19274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "978c155c77334f5bbff4c33d35c49284" candidate_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42af7a9808904218be2914ec3e665c2d"
I20260807 05:08:30.110237 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.104461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:35493 (local address 127.1.17.195:33687)
0807 05:08:30.105108 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.105121 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.105133 (+ 12us) server_negotiation.cc:407] Connection header received
0807 05:08:30.105171 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.105173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.105219 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.105310 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:30.105672 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.106612 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.108227 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.108897 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.109024 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.109068 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.109960 (+ 892us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.110056 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":581,"thread_start_us":76,"threads_started":1}
I20260807 05:08:30.110209 19135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "978c155c77334f5bbff4c33d35c49284" candidate_uuid: "874ab346d8434a36942370f8e9f8f3ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" is_pre_election: true
I20260807 05:08:30.110225 19274 raft_consensus.cc:3037] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:30.110435 19135 raft_consensus.cc:2379] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 874ab346d8434a36942370f8e9f8f3ed in current term 1: Already voted for candidate f6dfc37fb12844a48f11c3de4e60c19f in this term.
I20260807 05:08:30.110589 19273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "978c155c77334f5bbff4c33d35c49284" candidate_uuid: "874ab346d8434a36942370f8e9f8f3ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42af7a9808904218be2914ec3e665c2d" is_pre_election: true
I20260807 05:08:30.111038 18998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "978c155c77334f5bbff4c33d35c49284" candidate_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ab346d8434a36942370f8e9f8f3ed"
I20260807 05:08:30.111155 18998 raft_consensus.cc:3037] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:30.111150 18929 leader_election.cc:302] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 874ab346d8434a36942370f8e9f8f3ed; no voters: 42af7a9808904218be2914ec3e665c2d, f6dfc37fb12844a48f11c3de4e60c19f
I20260807 05:08:30.111644 19274 raft_consensus.cc:2454] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6dfc37fb12844a48f11c3de4e60c19f in term 1.
I20260807 05:08:30.111935 19069 leader_election.cc:302] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42af7a9808904218be2914ec3e665c2d, f6dfc37fb12844a48f11c3de4e60c19f; no voters:
I20260807 05:08:30.112113 19346 raft_consensus.cc:2781] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:30.112336 19346 raft_consensus.cc:686] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 1 LEADER]: Becoming Leader. State: Replica: f6dfc37fb12844a48f11c3de4e60c19f, State: Running, Role: LEADER
I20260807 05:08:30.112576 18998 raft_consensus.cc:2454] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6dfc37fb12844a48f11c3de4e60c19f in term 1.
I20260807 05:08:30.112699 19348 raft_consensus.cc:2726] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 05:08:30.112687 19346 consensus_queue.cc:237] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } }
I20260807 05:08:30.115923 18848 catalog_manager.cc:5495] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f reported cstate change: term changed from 0 to 1, leader changed from <none> to f6dfc37fb12844a48f11c3de4e60c19f (127.1.17.194). New cstate: current_term: 1 leader_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6dfc37fb12844a48f11c3de4e60c19f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35427 } health_report { overall_health: HEALTHY } } }
I20260807 05:08:30.197503 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.194012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39984 (local address 127.1.17.194:35427)
0807 05:08:30.194152 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.194156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.194170 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:08:30.194273 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.194275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.194330 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.194419 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:08:30.194646 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.195215 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.196072 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.196281 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.196358 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.196654 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:08:30.197121 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:08:30.197215 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.197277 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.197321 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:08:30.203771 19274 raft_consensus.cc:1261] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 1 FOLLOWER]: Refusing update from remote peer f6dfc37fb12844a48f11c3de4e60c19f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:30.203786 18998 raft_consensus.cc:1261] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 1 FOLLOWER]: Refusing update from remote peer f6dfc37fb12844a48f11c3de4e60c19f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:30.204438 19346 consensus_queue.cc:1035] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42af7a9808904218be2914ec3e665c2d" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 33687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:30.204643 19354 consensus_queue.cc:1035] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [LEADER]: Connected to new peer: Peer: permanent_uuid: "874ab346d8434a36942370f8e9f8f3ed" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 35225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:30.212747 19358 mvcc.cc:205] Tried to move back new op lower bound from 7315780854587023360 to 7315780854224510976. Current Snapshot: MvccSnapshot[applied={T|T < 7315780854587023360}]
I20260807 05:08:30.220670 18847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:30.220875 18847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:30.223448 18847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcb39eb8c9a34471a3e3834999ca302c: Sending DeleteTablet for 3 replicas of tablet 978c155c77334f5bbff4c33d35c49284
I20260807 05:08:30.224290 18977 tablet_service.cc:1514] Processing DeleteTablet for tablet 978c155c77334f5bbff4c33d35c49284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:30 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:50140
I20260807 05:08:30.224278 19115 tablet_service.cc:1514] Processing DeleteTablet for tablet 978c155c77334f5bbff4c33d35c49284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:30 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:39970
I20260807 05:08:30.224370 19253 tablet_service.cc:1514] Processing DeleteTablet for tablet 978c155c77334f5bbff4c33d35c49284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:30 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:34094
I20260807 05:08:30.224943 19368 tablet_replica.cc:330] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed: stopping tablet replica
I20260807 05:08:30.225267 19368 raft_consensus.cc:2229] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:30.225399 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 18910
I20260807 05:08:30.225606 19368 raft_consensus.cc:2258] T 978c155c77334f5bbff4c33d35c49284 P 874ab346d8434a36942370f8e9f8f3ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:30.225910 19367 tablet_replica.cc:330] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: stopping tablet replica
I20260807 05:08:30.226150 19367 raft_consensus.cc:2229] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:08:30.226440 19367 raft_consensus.cc:2258] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:30.226728 19369 tablet_replica.cc:330] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: stopping tablet replica
I20260807 05:08:30.226958 19369 raft_consensus.cc:2229] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:30.227188 19369 raft_consensus.cc:2258] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:30.228811 19367 ts_tablet_manager.cc:1905] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:30.231604 19367 ts_tablet_manager.cc:1918] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:30.231710 19367 log.cc:1196] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/978c155c77334f5bbff4c33d35c49284
I20260807 05:08:30.232041 19367 ts_tablet_manager.cc:1939] T 978c155c77334f5bbff4c33d35c49284 P f6dfc37fb12844a48f11c3de4e60c19f: Deleting consensus metadata
I20260807 05:08:30.233343 18833 catalog_manager.cc:4841] TS f6dfc37fb12844a48f11c3de4e60c19f (127.1.17.194:35427): tablet 978c155c77334f5bbff4c33d35c49284 (table test-table [id=90a1e62358f44d05989e532441e719ac]) successfully deleted
I20260807 05:08:30.233410 19369 ts_tablet_manager.cc:1905] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 05:08:30.235589 18834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:35225 (error 108)
I20260807 05:08:30.235860 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 19050
I20260807 05:08:30.238539 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.238263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:35225 (local address 127.0.0.1:50146)
0807 05:08:30.238395 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:30.238465 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 05:08:30.238755 18834 catalog_manager.cc:4584] TS 874ab346d8434a36942370f8e9f8f3ed (127.1.17.193:35225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 978c155c77334f5bbff4c33d35c49284: Network error: Client connection negotiation failed: client connection to 127.1.17.193:35225: connect: Connection refused (error 111)
I20260807 05:08:30.239558 19369 ts_tablet_manager.cc:1918] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:30.239666 19369 log.cc:1196] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/978c155c77334f5bbff4c33d35c49284
I20260807 05:08:30.239992 19369 ts_tablet_manager.cc:1939] T 978c155c77334f5bbff4c33d35c49284 P 42af7a9808904218be2914ec3e665c2d: Deleting consensus metadata
I20260807 05:08:30.241365 18834 catalog_manager.cc:4841] TS 42af7a9808904218be2914ec3e665c2d (127.1.17.195:33687): tablet 978c155c77334f5bbff4c33d35c49284 (table test-table [id=90a1e62358f44d05989e532441e719ac]) successfully deleted
I20260807 05:08:30.244194 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 19188
I20260807 05:08:30.250590 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 18815
2026-08-07T05:08:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2647 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T05:08:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:30Z Disabled control of system clock
I20260807 05:08:30.283241 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36347
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:36347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:08:30.393687 19376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:30.393953 19376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:30.394019 19376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:30.398084 19376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:30.398138 19376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:30.398154 19376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:30.398170 19376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:30.398226 19376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:30.402875 19376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:36347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:36347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:30.404074 19376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:30.405525 19376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:30.414033 19381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:30.414077 19384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:30.414149 19376 server_base.cc:1034] running on GCE node
W20260807 05:08:30.414047 19382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:30.414918 19376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:30.415974 19376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:30.417137 19376 hybrid_clock.cc:648] HybridClock initialized: now 1786079310417107 us; error 40 us; skew 500 ppm
I20260807 05:08:30.419095 19376 webserver.cc:460] Webserver started at http://127.1.17.254:39805/ using document root <none> and password file <none>
I20260807 05:08:30.419684 19376 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:30.419749 19376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:30.419989 19376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:30.422112 19376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "0ca8dd2c905a1074717b170a90af5b8f"
server_key_iv: "6011e0261acd7474c20f30d044741904"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.422689 19376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "0ca8dd2c905a1074717b170a90af5b8f"
server_key_iv: "6011e0261acd7474c20f30d044741904"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.426429 19376 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:08:30.428933 19390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:30.430197 19376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:30.430327 19376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "0ca8dd2c905a1074717b170a90af5b8f"
server_key_iv: "6011e0261acd7474c20f30d044741904"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.430449 19376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:30.443727 19376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:30.444403 19376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:30.444607 19376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:30.451705 19376 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:36347
I20260807 05:08:30.451714 19442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:36347 every 8 connection(s)
I20260807 05:08:30.452670 19376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:30.455827 19443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:30.459105 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 19376
I20260807 05:08:30.459268 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:30.459573 1095 external_mini_cluster.cc:1428] Setting key 2682f706ba703a5e5b513d20ba8571a5
I20260807 05:08:30.462087 19443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb: Bootstrap starting.
I20260807 05:08:30.464262 19443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:30.465095 19443 log.cc:824] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:30.466998 19443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb: No bootstrap required, opened a new log
I20260807 05:08:30.469240 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.461187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40276 (local address 127.1.17.254:36347)
0807 05:08:30.461836 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.461857 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.461920 (+ 63us) server_negotiation.cc:407] Connection header received
0807 05:08:30.462604 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.462622 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.463030 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.463369 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:30.463810 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.464914 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.465943 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.466441 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.466618 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.467488 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:30.467493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:30.467506 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:30.467898 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:30.467952 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:30.467972 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:30.468115 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.468242 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.468496 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":133,"threads_started":1}
I20260807 05:08:30.470770 19443 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36347 } }
I20260807 05:08:30.471046 19443 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:30.471097 19443 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa470bb9a7ee4e80a323dc27a3de9bbb, State: Initialized, Role: FOLLOWER
I20260807 05:08:30.471578 19443 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36347 } }
I20260807 05:08:30.471716 19443 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:30.471763 19443 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:30.471848 19443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:30.472632 19443 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36347 } }
I20260807 05:08:30.472971 19443 leader_election.cc:302] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa470bb9a7ee4e80a323dc27a3de9bbb; no voters:
I20260807 05:08:30.473212 19443 leader_election.cc:288] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:30.473338 19448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:30.473592 19448 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [term 1 LEADER]: Becoming Leader. State: Replica: aa470bb9a7ee4e80a323dc27a3de9bbb, State: Running, Role: LEADER
I20260807 05:08:30.473932 19448 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36347 } }
I20260807 05:08:30.474426 19443 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:30.475550 19450 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa470bb9a7ee4e80a323dc27a3de9bbb. Latest consensus state: current_term: 1 leader_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36347 } } }
I20260807 05:08:30.475698 19450 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:30.476034 19449 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa470bb9a7ee4e80a323dc27a3de9bbb" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 36347 } } }
I20260807 05:08:30.476100 19457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:30.476135 19449 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:30.479327 19457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:30.484493 19457 catalog_manager.cc:1347] Generated new cluster ID: 370c30c07b69485680d24134b0c231d3
I20260807 05:08:30.484555 19457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:30.490033 19457 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:30.490841 19457 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:30.499032 19457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb: Generated new TSK 0
I20260807 05:08:30.499805 19457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:30.508740 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36347
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:08:30.623827 19467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:30.624071 19467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:30.624107 19467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:30.627990 19467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:30.628038 19467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:30.628087 19467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:08:30.632576 19467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:30.633642 19467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:30.635123 19467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:30.644609 19473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:30.644646 19472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:30.644680 19467 server_base.cc:1034] running on GCE node
W20260807 05:08:30.644623 19475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:30.645234 19467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:30.645799 19467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:30.646960 19467 hybrid_clock.cc:648] HybridClock initialized: now 1786079310646948 us; error 24 us; skew 500 ppm
I20260807 05:08:30.648736 19467 webserver.cc:460] Webserver started at http://127.1.17.193:40955/ using document root <none> and password file <none>
I20260807 05:08:30.649343 19467 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:30.649405 19467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:30.649678 19467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:30.651664 19467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "d0bdeb278d89416f957ab4bf6df59ff5"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "aadeeab342c1c9174d0afdda60363a5a"
server_key_iv: "5d5b2b6ef97bc2a5949bfeb20bd2cb7f"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.652213 19467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "d0bdeb278d89416f957ab4bf6df59ff5"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "aadeeab342c1c9174d0afdda60363a5a"
server_key_iv: "5d5b2b6ef97bc2a5949bfeb20bd2cb7f"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.655920 19467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:08:30.658303 19481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:30.659435 19467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:08:30.659559 19467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "d0bdeb278d89416f957ab4bf6df59ff5"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "aadeeab342c1c9174d0afdda60363a5a"
server_key_iv: "5d5b2b6ef97bc2a5949bfeb20bd2cb7f"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.659659 19467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:30.676756 19467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:30.677717 19467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:30.677913 19467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:30.678524 19467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:30.679466 19467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:30.679539 19467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:30.679603 19467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:30.679633 19467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:30.688287 19467 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:42621
I20260807 05:08:30.688318 19595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:42621 every 8 connection(s)
I20260807 05:08:30.689271 19467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:08:30.696291 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 19467
I20260807 05:08:30.696413 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:08:30.696771 1095 external_mini_cluster.cc:1428] Setting key 80f4c09968ebe33d6720d7f04a1c1070
I20260807 05:08:30.698715 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.691051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:34673 (local address 127.1.17.254:36347)
0807 05:08:30.691191 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.691194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.692220 (+ 1026us) server_negotiation.cc:407] Connection header received
0807 05:08:30.693714 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.693717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.693777 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.693878 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:30.694753 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.695303 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.696714 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.697064 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.697162 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.698240 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:30.698244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:30.698247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:30.698263 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:30.698289 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:30.698299 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:30.698366 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.698556 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.698607 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:08:30.698913 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36347
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:08:30.699880 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.691444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36347 (local address 127.1.17.193:34673)
0807 05:08:30.692081 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:30.692121 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:30.693489 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:30.694058 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:30.694064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:30.694145 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:30.694612 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.694623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.695442 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.695451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.696568 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.696580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.697240 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.697244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.697393 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.698002 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:30.698003 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:30.698042 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:30.698102 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:30.698394 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:30.698404 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:30.698411 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 05:08:30.698550 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 05:08:30.698840 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":116,"threads_started":1}
I20260807 05:08:30.701109 19596 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36347
I20260807 05:08:30.701362 19596 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:30.701969 19596 heartbeater.cc:511] Master 127.1.17.254:36347 requested a full tablet report, sending...
I20260807 05:08:30.703540 19407 ts_manager.cc:194] Registered new tserver with Master: d0bdeb278d89416f957ab4bf6df59ff5 (127.1.17.193:42621)
I20260807 05:08:30.704916 19407 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.193:34673
W20260807 05:08:30.813498 19599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:30.813730 19599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:30.813767 19599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:30.817751 19599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:30.817800 19599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:30.817893 19599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:08:30.822566 19599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:30.823613 19599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:30.825016 19599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:30.836794 19608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:30.836853 19605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:30.836803 19606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:30.837029 19599 server_base.cc:1034] running on GCE node
I20260807 05:08:30.837505 19599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:30.838145 19599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:30.839335 19599 hybrid_clock.cc:648] HybridClock initialized: now 1786079310839320 us; error 37 us; skew 500 ppm
I20260807 05:08:30.841267 19599 webserver.cc:460] Webserver started at http://127.1.17.194:34305/ using document root <none> and password file <none>
I20260807 05:08:30.841852 19599 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:30.841953 19599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:30.842168 19599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:30.844218 19599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "b1de43615bd4e9b10386ea491d6ed3fc"
server_key_iv: "1627937437173273e8950290d86c66ec"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.844851 19599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "b1de43615bd4e9b10386ea491d6ed3fc"
server_key_iv: "1627937437173273e8950290d86c66ec"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.848472 19599 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:08:30.850875 19614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:30.852020 19599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:30.852162 19599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6"
format_stamp: "Formatted at 2026-08-07 05:08:30 on dist-test-slave-1x4g"
server_key: "b1de43615bd4e9b10386ea491d6ed3fc"
server_key_iv: "1627937437173273e8950290d86c66ec"
server_key_version: "encryptionkey@0"
I20260807 05:08:30.852273 19599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:30.863498 19599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:30.864113 19599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:30.864295 19599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:30.864847 19599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:30.865710 19599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:30.865767 19599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:30.865800 19599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:30.865823 19599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:30.874564 19599 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:46439
I20260807 05:08:30.874595 19737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:46439 every 8 connection(s)
I20260807 05:08:30.875466 19599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:08:30.877066 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 19599
I20260807 05:08:30.877203 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:08:30.877518 1095 external_mini_cluster.cc:1428] Setting key 9bf4694b71fec39b29acc0633744f9d6
I20260807 05:08:30.879761 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36347
--builtin_ntp_servers=127.1.17.212:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:08:30.886029 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.878936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:59773 (local address 127.1.17.254:36347)
0807 05:08:30.879085 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:30.879090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:30.879108 (+ 18us) server_negotiation.cc:407] Connection header received
0807 05:08:30.880265 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:30.880268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:30.880319 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:30.880416 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:30.881375 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.881908 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.883499 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.883898 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.884023 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.885442 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:30.885452 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:30.885455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:30.885468 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:30.885490 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:30.885500 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:30.885585 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:30.885803 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 05:08:30.885879 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:08:30.887138 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:30.877938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36347 (local address 127.1.17.194:59773)
0807 05:08:30.878636 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:30.878675 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:30.880061 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:30.880586 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:30.880592 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:30.880693 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:30.881227 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.881240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.882045 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.882048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.883365 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:30.883377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:30.884114 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:30.884121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:30.884351 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:30.885087 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:30.885089 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:30.885146 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:30.885248 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:30.885644 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:30.885654 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:30.885660 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 05:08:30.885802 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 05:08:30.886069 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":112,"threads_started":1}
I20260807 05:08:30.889998 19738 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36347
I20260807 05:08:30.890223 19738 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:30.890853 19738 heartbeater.cc:511] Master 127.1.17.254:36347 requested a full tablet report, sending...
I20260807 05:08:30.892129 19407 ts_manager.cc:194] Registered new tserver with Master: e8a51bb0771f448bbc2766ef4eb8b6e6 (127.1.17.194:46439)
I20260807 05:08:30.892753 19407 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.194:59773
W20260807 05:08:30.994700 19742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:30.994937 19742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:30.994972 19742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:30.999014 19742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:30.999063 19742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:30.999152 19742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:08:31.003793 19742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:36347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:31.004917 19742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:31.006313 19742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:31.015828 19747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:31.015842 19748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:31.015849 19750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:31.016266 19742 server_base.cc:1034] running on GCE node
I20260807 05:08:31.016664 19742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:31.017268 19742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:31.018455 19742 hybrid_clock.cc:648] HybridClock initialized: now 1786079311018423 us; error 45 us; skew 500 ppm
I20260807 05:08:31.020183 19742 webserver.cc:460] Webserver started at http://127.1.17.195:43263/ using document root <none> and password file <none>
I20260807 05:08:31.020761 19742 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:31.020913 19742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:31.021148 19742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:31.023152 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "4203a26d79264b15a1b100dea4086309"
format_stamp: "Formatted at 2026-08-07 05:08:31 on dist-test-slave-1x4g"
server_key: "85d2f9679a8b38fd2b4c7ddf710e7b8e"
server_key_iv: "810d6f8f5574c6fba0cf907b14aeeb2c"
server_key_version: "encryptionkey@0"
I20260807 05:08:31.023703 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "4203a26d79264b15a1b100dea4086309"
format_stamp: "Formatted at 2026-08-07 05:08:31 on dist-test-slave-1x4g"
server_key: "85d2f9679a8b38fd2b4c7ddf710e7b8e"
server_key_iv: "810d6f8f5574c6fba0cf907b14aeeb2c"
server_key_version: "encryptionkey@0"
I20260807 05:08:31.027486 19742 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 05:08:31.029760 19757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:31.030879 19742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:31.031013 19742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "4203a26d79264b15a1b100dea4086309"
format_stamp: "Formatted at 2026-08-07 05:08:31 on dist-test-slave-1x4g"
server_key: "85d2f9679a8b38fd2b4c7ddf710e7b8e"
server_key_iv: "810d6f8f5574c6fba0cf907b14aeeb2c"
server_key_version: "encryptionkey@0"
I20260807 05:08:31.031121 19742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:31.053400 19742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:31.054086 19742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:31.054278 19742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:31.054845 19742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:31.055845 19742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:31.055915 19742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:31.055979 19742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:31.056033 19742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:31.064700 19742 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:39501
I20260807 05:08:31.064740 19875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:39501 every 8 connection(s)
I20260807 05:08:31.065831 19742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:08:31.067499 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 19742
I20260807 05:08:31.067620 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:08:31.067965 1095 external_mini_cluster.cc:1428] Setting key aff8d34db0a112d7016657f55b2451a4
I20260807 05:08:31.075130 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.067704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:58463 (local address 127.1.17.254:36347)
0807 05:08:31.067873 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.067877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.069157 (+ 1280us) server_negotiation.cc:407] Connection header received
0807 05:08:31.070644 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.070646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.070709 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.070782 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:31.071586 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.072129 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.073329 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.073554 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.073632 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.074624 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:31.074630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:31.074633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:31.074649 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:31.074676 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:31.074686 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:31.074744 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.074972 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.075022 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 05:08:31.076038 19878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.068269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:36347 (local address 127.1.17.195:58463)
0807 05:08:31.069005 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:31.069057 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:31.070465 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:31.070912 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:31.070918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:31.071000 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:31.071472 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.071484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.072232 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.072235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.073224 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.073234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.073696 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.073699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.073837 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.074365 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:31.074367 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:31.074405 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:31.074472 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:31.074826 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:31.074838 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:31.074845 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 05:08:31.074954 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 05:08:31.075162 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":138,"threads_started":1}
I20260807 05:08:31.077154 19876 heartbeater.cc:347] Connected to a master server at 127.1.17.254:36347
I20260807 05:08:31.077348 19876 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:31.077781 19876 heartbeater.cc:511] Master 127.1.17.254:36347 requested a full tablet report, sending...
I20260807 05:08:31.078735 19407 ts_manager.cc:194] Registered new tserver with Master: 4203a26d79264b15a1b100dea4086309 (127.1.17.195:39501)
I20260807 05:08:31.079306 19407 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.195:58463
I20260807 05:08:31.081787 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:08:31.087077 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.083736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.1.17.254:36347)
0807 05:08:31.083866 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.083872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.084013 (+ 141us) server_negotiation.cc:407] Connection header received
0807 05:08:31.084088 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.084090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.084124 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.084202 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:31.084414 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.084949 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.085761 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.085959 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.086030 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.086703 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:31.086705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:31.086708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:31.086717 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:31.086734 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:31.086741 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:31.086795 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.086867 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.086920 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 05:08:31.089516 19407 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:08:31.092427 19407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:08:31.105990 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.100353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:39501 (local address 127.0.0.1:57472)
0807 05:08:31.101148 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:31.101196 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:31.101324 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:31.101964 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:31.101970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:31.101992 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:31.102130 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.102136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.103344 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.103347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.104602 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.104611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.105494 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.105498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.105602 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.105640 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 05:08:31.105741 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 05:08:31.105850 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"spinlock_wait_cycles":2432,"thread_start_us":120,"threads_started":1}
I20260807 05:08:31.106179 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.100543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.1.17.195:39501)
0807 05:08:31.100955 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.100967 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.101490 (+ 523us) server_negotiation.cc:407] Connection header received
0807 05:08:31.101572 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.101577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.101777 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.101968 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:31.102242 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.103229 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.104720 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.105377 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.105480 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.105646 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.105786 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.105892 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":150,"threads_started":1}
I20260807 05:08:31.106564 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.100841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:42621 (local address 127.0.0.1:58288)
0807 05:08:31.101902 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:31.101916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:31.102033 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:31.102536 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:31.102539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:31.102557 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:31.102646 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.102651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.103771 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.103775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.104987 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.104998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.106181 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.106185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.106296 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.106323 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 05:08:31.106396 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:08:31.106469 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":35,"spinlock_wait_cycles":11008,"thread_start_us":92,"threads_started":1}
I20260807 05:08:31.106899 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.101133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:46439 (local address 127.0.0.1:57894)
0807 05:08:31.101902 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:31.101916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:31.102032 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:31.102686 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:31.102689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:31.102730 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:31.102821 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.102827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.103983 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.103986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.105086 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.105095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.106534 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.106538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.106622 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.106643 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:08:31.106712 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 05:08:31.106775 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":19,"spinlock_wait_cycles":19328,"thread_start_us":100,"threads_started":1}
I20260807 05:08:31.107055 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.101291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.1.17.193:42621)
0807 05:08:31.101746 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.101760 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.102134 (+ 374us) server_negotiation.cc:407] Connection header received
0807 05:08:31.102218 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.102223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.102390 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.102543 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:31.102773 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.103654 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.105106 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.105818 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.105905 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.106106 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.106805 (+ 699us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.106889 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":147,"threads_started":1}
I20260807 05:08:31.107748 19893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.101320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.1.17.194:46439)
0807 05:08:31.101758 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.101769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.102136 (+ 367us) server_negotiation.cc:407] Connection header received
0807 05:08:31.102218 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.102223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.102390 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.102543 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:31.102957 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.103844 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.105211 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.105955 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.106071 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.106468 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.107129 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.107599 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":147,"threads_started":1}
I20260807 05:08:31.109218 19806 tablet_service.cc:1467] Processing CreateTablet for tablet b44b6c0621f44ff9b146fe32edf44928 (DEFAULT_TABLE table=test-table [id=9c476f06ad6e4a77ae76740aafded67f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:31.109624 19530 tablet_service.cc:1467] Processing CreateTablet for tablet b44b6c0621f44ff9b146fe32edf44928 (DEFAULT_TABLE table=test-table [id=9c476f06ad6e4a77ae76740aafded67f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:31.110199 19806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44b6c0621f44ff9b146fe32edf44928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:31.110635 19530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44b6c0621f44ff9b146fe32edf44928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:31.110258 19666 tablet_service.cc:1467] Processing CreateTablet for tablet b44b6c0621f44ff9b146fe32edf44928 (DEFAULT_TABLE table=test-table [id=9c476f06ad6e4a77ae76740aafded67f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:31.111229 19666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44b6c0621f44ff9b146fe32edf44928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:31.116053 19894 tablet_bootstrap.cc:492] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Bootstrap starting.
I20260807 05:08:31.117712 19895 tablet_bootstrap.cc:492] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5: Bootstrap starting.
I20260807 05:08:31.118458 19894 tablet_bootstrap.cc:654] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:31.118968 19896 tablet_bootstrap.cc:492] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Bootstrap starting.
I20260807 05:08:31.119410 19894 log.cc:824] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:31.119876 19895 tablet_bootstrap.cc:654] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:31.120703 19895 log.cc:824] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:31.120750 19896 tablet_bootstrap.cc:654] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:31.121543 19896 log.cc:824] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:31.121562 19894 tablet_bootstrap.cc:492] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: No bootstrap required, opened a new log
I20260807 05:08:31.121783 19894 ts_tablet_manager.cc:1397] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:08:31.122818 19895 tablet_bootstrap.cc:492] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5: No bootstrap required, opened a new log
I20260807 05:08:31.123004 19895 ts_tablet_manager.cc:1397] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 05:08:31.126017 19896 tablet_bootstrap.cc:492] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: No bootstrap required, opened a new log
I20260807 05:08:31.126200 19896 ts_tablet_manager.cc:1397] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 05:08:31.126266 19894 raft_consensus.cc:348] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.126524 19894 raft_consensus.cc:374] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:31.126582 19894 raft_consensus.cc:729] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4203a26d79264b15a1b100dea4086309, State: Initialized, Role: FOLLOWER
I20260807 05:08:31.127030 19894 consensus_queue.cc:260] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.127128 19895 raft_consensus.cc:348] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.127377 19895 raft_consensus.cc:374] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:31.127444 19895 raft_consensus.cc:729] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bdeb278d89416f957ab4bf6df59ff5, State: Initialized, Role: FOLLOWER
I20260807 05:08:31.127781 19876 heartbeater.cc:503] Master 127.1.17.254:36347 was elected leader, sending a full tablet report...
I20260807 05:08:31.127861 19895 consensus_queue.cc:260] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.128762 19596 heartbeater.cc:503] Master 127.1.17.254:36347 was elected leader, sending a full tablet report...
I20260807 05:08:31.128763 19895 ts_tablet_manager.cc:1428] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260807 05:08:31.130230 19896 raft_consensus.cc:348] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.130429 19896 raft_consensus.cc:374] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:31.130505 19896 raft_consensus.cc:729] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a51bb0771f448bbc2766ef4eb8b6e6, State: Initialized, Role: FOLLOWER
I20260807 05:08:31.130863 19894 ts_tablet_manager.cc:1428] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 05:08:31.130918 19896 consensus_queue.cc:260] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.131517 19738 heartbeater.cc:503] Master 127.1.17.254:36347 was elected leader, sending a full tablet report...
I20260807 05:08:31.131839 19896 ts_tablet_manager.cc:1428] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 05:08:31.190475 19597 tablet.cc:2369] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:08:31.223330 19901 raft_consensus.cc:482] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:31.223559 19901 raft_consensus.cc:504] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.224853 19901 leader_election.cc:288] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4203a26d79264b15a1b100dea4086309 (127.1.17.195:39501), e8a51bb0771f448bbc2766ef4eb8b6e6 (127.1.17.194:46439)
I20260807 05:08:31.228649 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.225236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:33667 (local address 127.1.17.195:39501)
0807 05:08:31.225362 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.225366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.225454 (+ 88us) server_negotiation.cc:407] Connection header received
0807 05:08:31.225691 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.225694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.225756 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.225840 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:31.226158 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.226669 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.227723 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.228128 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.228247 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.228299 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.228476 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.228542 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 05:08:31.228665 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.225052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:39501 (local address 127.1.17.193:33667)
0807 05:08:31.225363 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:31.225380 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:31.225536 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:31.225875 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:31.225879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:31.225900 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:31.226053 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.226060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.226782 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.226788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.227593 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.227602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.228248 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.228251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.228338 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.228365 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 05:08:31.228450 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 05:08:31.228515 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":129,"threads_started":1}
I20260807 05:08:31.229031 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.225485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:46439 (local address 127.1.17.193:42225)
0807 05:08:31.225896 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:31.225910 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:31.225996 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:31.226254 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:31.226257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:31.226274 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:31.226403 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.226414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.227139 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.227144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.227907 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:31.227916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.228534 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.228537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:31.228632 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.228653 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:08:31.228747 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 05:08:31.228890 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":97,"threads_started":1}
I20260807 05:08:31.229140 19893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.225603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:42225 (local address 127.1.17.194:46439)
0807 05:08:31.225734 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.225739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.225978 (+ 239us) server_negotiation.cc:407] Connection header received
0807 05:08:31.226095 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.226098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.226140 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.226228 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:31.226541 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.227027 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.228038 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.228403 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.228533 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.228578 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.228942 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.229014 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:08:31.229293 19828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44b6c0621f44ff9b146fe32edf44928" candidate_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4203a26d79264b15a1b100dea4086309" is_pre_election: true
I20260807 05:08:31.229626 19828 raft_consensus.cc:2454] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0bdeb278d89416f957ab4bf6df59ff5 in term 0.
I20260807 05:08:31.229648 19688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44b6c0621f44ff9b146fe32edf44928" candidate_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" is_pre_election: true
I20260807 05:08:31.229914 19688 raft_consensus.cc:2454] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0bdeb278d89416f957ab4bf6df59ff5 in term 0.
I20260807 05:08:31.230108 19482 leader_election.cc:302] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4203a26d79264b15a1b100dea4086309, d0bdeb278d89416f957ab4bf6df59ff5; no voters:
I20260807 05:08:31.230393 19901 raft_consensus.cc:2781] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:08:31.230473 19901 raft_consensus.cc:482] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:08:31.230504 19901 raft_consensus.cc:3037] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:31.231554 19901 raft_consensus.cc:504] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.231880 19901 leader_election.cc:288] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [CANDIDATE]: Term 1 election: Requested vote from peers 4203a26d79264b15a1b100dea4086309 (127.1.17.195:39501), e8a51bb0771f448bbc2766ef4eb8b6e6 (127.1.17.194:46439)
I20260807 05:08:31.232250 19828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44b6c0621f44ff9b146fe32edf44928" candidate_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4203a26d79264b15a1b100dea4086309"
I20260807 05:08:31.232267 19688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44b6c0621f44ff9b146fe32edf44928" candidate_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6"
I20260807 05:08:31.232352 19828 raft_consensus.cc:3037] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:31.232400 19688 raft_consensus.cc:3037] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:31.233464 19828 raft_consensus.cc:2454] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0bdeb278d89416f957ab4bf6df59ff5 in term 1.
I20260807 05:08:31.233780 19688 raft_consensus.cc:2454] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0bdeb278d89416f957ab4bf6df59ff5 in term 1.
I20260807 05:08:31.233950 19482 leader_election.cc:302] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4203a26d79264b15a1b100dea4086309, d0bdeb278d89416f957ab4bf6df59ff5; no voters:
I20260807 05:08:31.234091 19901 raft_consensus.cc:2781] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:31.234354 19901 raft_consensus.cc:686] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [term 1 LEADER]: Becoming Leader. State: Replica: d0bdeb278d89416f957ab4bf6df59ff5, State: Running, Role: LEADER
I20260807 05:08:31.234720 19901 consensus_queue.cc:237] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } }
I20260807 05:08:31.237819 19407 catalog_manager.cc:5495] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0bdeb278d89416f957ab4bf6df59ff5 (127.1.17.193). New cstate: current_term: 1 leader_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0bdeb278d89416f957ab4bf6df59ff5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 42621 } health_report { overall_health: HEALTHY } } }
I20260807 05:08:31.263303 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.259628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.1.17.193:42621)
0807 05:08:31.259770 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:31.259775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:31.259789 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:08:31.259988 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:31.259990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:31.260043 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:31.260155 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:08:31.260423 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.260962 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.261851 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:31.262075 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:31.262181 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:31.262520 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:08:31.262920 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:08:31.263054 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:31.263118 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 05:08:31.263167 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:08:31.269469 19828 raft_consensus.cc:1261] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 1 FOLLOWER]: Refusing update from remote peer d0bdeb278d89416f957ab4bf6df59ff5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:31.270188 19905 consensus_queue.cc:1035] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4203a26d79264b15a1b100dea4086309" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:31.270359 19688 raft_consensus.cc:1261] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 1 FOLLOWER]: Refusing update from remote peer d0bdeb278d89416f957ab4bf6df59ff5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:31.270991 19905 consensus_queue.cc:1035] T b44b6c0621f44ff9b146fe32edf44928 P d0bdeb278d89416f957ab4bf6df59ff5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a51bb0771f448bbc2766ef4eb8b6e6" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:31.278535 19910 mvcc.cc:205] Tried to move back new op lower bound from 7315780858952626176 to 7315780858820284416. Current Snapshot: MvccSnapshot[applied={T|T < 7315780858952626176}]
I20260807 05:08:31.280508 19909 mvcc.cc:205] Tried to move back new op lower bound from 7315780858952626176 to 7315780858820284416. Current Snapshot: MvccSnapshot[applied={T|T < 7315780858952626176}]
I20260807 05:08:31.286798 19406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:31.286965 19406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:31.289464 19406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa470bb9a7ee4e80a323dc27a3de9bbb: Sending DeleteTablet for 3 replicas of tablet b44b6c0621f44ff9b146fe32edf44928
I20260807 05:08:31.290314 19806 tablet_service.cc:1514] Processing DeleteTablet for tablet b44b6c0621f44ff9b146fe32edf44928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:31 UTC) from {username='slave'} at 127.0.0.1:57472
I20260807 05:08:31.290318 19666 tablet_service.cc:1514] Processing DeleteTablet for tablet b44b6c0621f44ff9b146fe32edf44928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:31 UTC) from {username='slave'} at 127.0.0.1:57894
I20260807 05:08:31.290315 19530 tablet_service.cc:1514] Processing DeleteTablet for tablet b44b6c0621f44ff9b146fe32edf44928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:31 UTC) from {username='slave'} at 127.0.0.1:58288
I20260807 05:08:31.290838 19921 tablet_replica.cc:330] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: stopping tablet replica
I20260807 05:08:31.291143 19921 raft_consensus.cc:2229] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:31.291328 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 19467
I20260807 05:08:31.294047 19921 raft_consensus.cc:2258] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:31.295456 19921 ts_tablet_manager.cc:1905] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:31.296975 19919 tablet_replica.cc:330] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: stopping tablet replica
I20260807 05:08:31.297257 19919 raft_consensus.cc:2229] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:31.297537 19919 raft_consensus.cc:2258] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:31.298724 19919 ts_tablet_manager.cc:1905] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:31.298944 19921 ts_tablet_manager.cc:1918] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:31.299054 19921 log.cc:1196] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/b44b6c0621f44ff9b146fe32edf44928
I20260807 05:08:31.299418 19921 ts_tablet_manager.cc:1939] T b44b6c0621f44ff9b146fe32edf44928 P 4203a26d79264b15a1b100dea4086309: Deleting consensus metadata
I20260807 05:08:31.300686 19391 catalog_manager.cc:4841] TS 4203a26d79264b15a1b100dea4086309 (127.1.17.195:39501): tablet b44b6c0621f44ff9b146fe32edf44928 (table test-table [id=9c476f06ad6e4a77ae76740aafded67f]) successfully deleted
I20260807 05:08:31.301164 19919 ts_tablet_manager.cc:1918] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:31.301275 19919 log.cc:1196] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/b44b6c0621f44ff9b146fe32edf44928
I20260807 05:08:31.301628 19919 ts_tablet_manager.cc:1939] T b44b6c0621f44ff9b146fe32edf44928 P e8a51bb0771f448bbc2766ef4eb8b6e6: Deleting consensus metadata
W20260807 05:08:31.301788 19393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:42621 (error 108)
I20260807 05:08:31.302589 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 19599
I20260807 05:08:31.302850 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:31.302559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:42621 (local address 127.0.0.1:58302)
0807 05:08:31.302687 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:31.302760 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:42621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 05:08:31.303714 19393 catalog_manager.cc:4584] TS d0bdeb278d89416f957ab4bf6df59ff5 (127.1.17.193:42621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b44b6c0621f44ff9b146fe32edf44928: Network error: Client connection negotiation failed: client connection to 127.1.17.193:42621: connect: Connection refused (error 111)
I20260807 05:08:31.303751 19391 catalog_manager.cc:4841] TS e8a51bb0771f448bbc2766ef4eb8b6e6 (127.1.17.194:46439): tablet b44b6c0621f44ff9b146fe32edf44928 (table test-table [id=9c476f06ad6e4a77ae76740aafded67f]) successfully deleted
I20260807 05:08:31.308934 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 19742
I20260807 05:08:31.315709 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 19376
2026-08-07T05:08:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1063 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:31 dist-test-slave-1x4g krb5kdc[19925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:31 dist-test-slave-1x4g krb5kdc[19925](info): set up 2 sockets
Aug 07 05:08:31 dist-test-slave-1x4g krb5kdc[19925](info): commencing operation
krb5kdc: starting...
W20260807 05:08:32.357538 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:32 dist-test-slave-1x4g krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:08:32.477833 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43611
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:32.587152 19939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:32.587388 19939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:32.587425 19939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:32.591289 19939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:32.591341 19939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:32.591360 19939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:32.591377 19939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:32.591391 19939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:32.596457 19939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:32.597484 19939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:32.598492 19939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:32.607031 19944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:32.607079 19947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:32.607190 19939 server_base.cc:1034] running on GCE node
W20260807 05:08:32.607031 19945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:32.607851 19939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:32.609081 19939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:32.610280 19939 hybrid_clock.cc:648] HybridClock initialized: now 1786079312610246 us; error 44 us; skew 500 ppm
Aug 07 05:08:32 dist-test-slave-1x4g krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079312, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:32.613286 19939 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:08:32.614190 19939 webserver.cc:460] Webserver started at http://127.1.17.254:45945/ using document root <none> and password file <none>
I20260807 05:08:32.614786 19939 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:32.614884 19939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:32.615161 19939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:32.617419 19939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "f40bfba33fdd4cf783f5080c38300194"
format_stamp: "Formatted at 2026-08-07 05:08:32 on dist-test-slave-1x4g"
server_key: "6de736b151ab8e31b4351afe8217e8af"
server_key_iv: "0fb3e431fec22bfdd5060f687ef0885c"
server_key_version: "encryptionkey@0"
I20260807 05:08:32.618050 19939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "f40bfba33fdd4cf783f5080c38300194"
format_stamp: "Formatted at 2026-08-07 05:08:32 on dist-test-slave-1x4g"
server_key: "6de736b151ab8e31b4351afe8217e8af"
server_key_iv: "0fb3e431fec22bfdd5060f687ef0885c"
server_key_version: "encryptionkey@0"
I20260807 05:08:32.621989 19939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 05:08:32.624611 19954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:32.625890 19939 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:08:32.626067 19939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "f40bfba33fdd4cf783f5080c38300194"
format_stamp: "Formatted at 2026-08-07 05:08:32 on dist-test-slave-1x4g"
server_key: "6de736b151ab8e31b4351afe8217e8af"
server_key_iv: "0fb3e431fec22bfdd5060f687ef0885c"
server_key_version: "encryptionkey@0"
I20260807 05:08:32.626189 19939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:32.645326 19939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:32.648252 19939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:32.648468 19939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:32.656432 19939 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:43611
I20260807 05:08:32.656452 20009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:43611 every 8 connection(s)
I20260807 05:08:32.657497 19939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:32.660324 20010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:32.663712 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 19939
I20260807 05:08:32.663904 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:32.664276 1095 external_mini_cluster.cc:1428] Setting key 47cd1c9b7b81a41b9e1f30d4a83dc285
I20260807 05:08:32.666445 20010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194: Bootstrap starting.
I20260807 05:08:32.669157 20010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:32.670032 20010 log.cc:824] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:32.672187 20010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194: No bootstrap required, opened a new log
Aug 07 05:08:32 dist-test-slave-1x4g krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:32.676659 20010 raft_consensus.cc:348] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40bfba33fdd4cf783f5080c38300194" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43611 } }
I20260807 05:08:32.676944 20010 raft_consensus.cc:374] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:32.677026 20010 raft_consensus.cc:729] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40bfba33fdd4cf783f5080c38300194, State: Initialized, Role: FOLLOWER
I20260807 05:08:32.677511 20010 consensus_queue.cc:260] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40bfba33fdd4cf783f5080c38300194" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43611 } }
I20260807 05:08:32.677667 20010 raft_consensus.cc:388] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:32.677753 20010 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:32.677871 20010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:32.678985 20010 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40bfba33fdd4cf783f5080c38300194" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43611 } }
I20260807 05:08:32.679330 20010 leader_election.cc:302] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f40bfba33fdd4cf783f5080c38300194; no voters:
I20260807 05:08:32.679641 20010 leader_election.cc:288] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:32.679762 20015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:32.679983 20015 raft_consensus.cc:686] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [term 1 LEADER]: Becoming Leader. State: Replica: f40bfba33fdd4cf783f5080c38300194, State: Running, Role: LEADER
I20260807 05:08:32.680125 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:32.665780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.1.17.254:43611)
0807 05:08:32.666209 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:32.666227 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:32.666268 (+ 41us) server_negotiation.cc:407] Connection header received
0807 05:08:32.667089 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:32.667112 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:32.667551 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:32.667910 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:32.668584 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:32.670163 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:32.671113 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:32.671780 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:32.671945 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:32.674593 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:32.674599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:32.674616 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:32.674652 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:32.676678 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:32.677273 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:32.677278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:32.677284 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:32.677436 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:32.677806 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:32.677809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:32.677811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:32.678308 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:32.678555 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:32.678850 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 05:08:32.679148 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":113,"threads_started":1}
I20260807 05:08:32.680290 20015 consensus_queue.cc:237] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40bfba33fdd4cf783f5080c38300194" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43611 } }
I20260807 05:08:32.680850 20010 sys_catalog.cc:564] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:32.681754 20017 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f40bfba33fdd4cf783f5080c38300194. Latest consensus state: current_term: 1 leader_uuid: "f40bfba33fdd4cf783f5080c38300194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40bfba33fdd4cf783f5080c38300194" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43611 } } }
I20260807 05:08:32.681891 20017 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:32.681902 20016 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f40bfba33fdd4cf783f5080c38300194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40bfba33fdd4cf783f5080c38300194" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43611 } } }
I20260807 05:08:32.682117 20016 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194 [sys.catalog]: This master's current role is: LEADER
W20260807 05:08:32.686017 20031 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:32.686100 20031 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:32.686362 20025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:32.687114 20025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:32.693409 20025 catalog_manager.cc:1347] Generated new cluster ID: 04b781f3711c4dfea10ac31c40c895d3
I20260807 05:08:32.693472 20025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:32.708324 20025 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:32.709486 20025 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:32.720924 20025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194: Generated new TSK 0
I20260807 05:08:32.721685 20025 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:08:32.755365 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43611
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:32.869923 20037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:32.870177 20037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:32.870263 20037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:32.875873 20037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:32.875972 20037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:32.876080 20037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:08:32.885344 20037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:32.886811 20037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:32.888195 20037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:32.900410 20043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:32.900383 20046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:32.901074 20042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:32.901460 20037 server_base.cc:1034] running on GCE node
I20260807 05:08:32.901914 20037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:32.902647 20037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:32.903859 20037 hybrid_clock.cc:648] HybridClock initialized: now 1786079312903841 us; error 36 us; skew 500 ppm
Aug 07 05:08:32 dist-test-slave-1x4g krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079312, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:32.907344 20037 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:08:32.908521 20037 webserver.cc:460] Webserver started at http://127.1.17.193:36159/ using document root <none> and password file <none>
I20260807 05:08:32.909369 20037 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:32.909487 20037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:32.909790 20037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:32.912977 20037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "259f4b44c4634262b7bbc25e3b52b3b5"
format_stamp: "Formatted at 2026-08-07 05:08:32 on dist-test-slave-1x4g"
server_key: "5204bd45dd0a0c532f3d2906d11fd7ad"
server_key_iv: "50e0e081973bfe5e231769e1cf88c83a"
server_key_version: "encryptionkey@0"
I20260807 05:08:32.913790 20037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "259f4b44c4634262b7bbc25e3b52b3b5"
format_stamp: "Formatted at 2026-08-07 05:08:32 on dist-test-slave-1x4g"
server_key: "5204bd45dd0a0c532f3d2906d11fd7ad"
server_key_iv: "50e0e081973bfe5e231769e1cf88c83a"
server_key_version: "encryptionkey@0"
I20260807 05:08:32.919381 20037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 05:08:32.922941 20053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:32.924537 20037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:08:32.924680 20037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "259f4b44c4634262b7bbc25e3b52b3b5"
format_stamp: "Formatted at 2026-08-07 05:08:32 on dist-test-slave-1x4g"
server_key: "5204bd45dd0a0c532f3d2906d11fd7ad"
server_key_iv: "50e0e081973bfe5e231769e1cf88c83a"
server_key_version: "encryptionkey@0"
I20260807 05:08:32.924870 20037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:32.945003 20037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:32.948165 20037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:32.948315 20037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:32.948876 20037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:32.949641 20037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:32.949694 20037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:32.949733 20037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:32.949754 20037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:32.958209 20037 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:44631
I20260807 05:08:32.958287 20171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:44631 every 8 connection(s)
I20260807 05:08:32.959120 20037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:08:32.962319 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 20037
I20260807 05:08:32.962461 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:08:32.962818 1095 external_mini_cluster.cc:1428] Setting key 782e976ff72026790517032cfb35fd87
Aug 07 05:08:32 dist-test-slave-1x4g krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079312, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:32.973616 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:32.960900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:41349 (local address 127.1.17.254:43611)
0807 05:08:32.961059 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:32.961064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:32.961948 (+ 884us) server_negotiation.cc:407] Connection header received
0807 05:08:32.963032 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:32.963036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:32.963093 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:32.963201 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:32.964504 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:32.965107 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:32.966451 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:32.966696 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:32.966782 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:32.969555 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:32.969558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:32.969560 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:32.969583 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:32.971350 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:32.971852 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:32.971855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:32.971857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:32.971952 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:32.972294 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:32.972298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:32.972301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:32.972578 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:32.972688 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:32.973230 (+ 542us) server_negotiation.cc:299] Negotiation successful
0807 05:08:32.973377 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:08:32.974390 20175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:32.961347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43611 (local address 127.1.17.193:41349)
0807 05:08:32.961791 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:32.961834 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:32.962770 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:32.963403 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:32.963412 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:32.963800 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:32.964362 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:32.964376 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:32.965253 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:32.965256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:32.966320 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:32.966335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:32.966856 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:32.966861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:32.967099 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:32.967733 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:32.967749 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:32.969374 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:32.971477 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:32.971481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:32.971492 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:32.971747 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:32.972065 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:32.972067 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:32.972068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:32.972195 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:32.972687 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:32.972692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:32.972973 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 05:08:32.973138 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 05:08:32.973388 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":104,"threads_started":1}
I20260807 05:08:32.975679 20173 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43611
I20260807 05:08:32.975960 20173 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:32.976537 20173 heartbeater.cc:511] Master 127.1.17.254:43611 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:08:32.978256 19971 ts_manager.cc:194] Registered new tserver with Master: 259f4b44c4634262b7bbc25e3b52b3b5 (127.1.17.193:44631)
I20260807 05:08:32.979321 19971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:41349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:08:32.991758 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43611
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:33.103827 20178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:33.104058 20178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:33.104097 20178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:33.108044 20178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:33.108090 20178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:33.108188 20178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:08:33.113477 20178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:33.114439 20178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:33.115417 20178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:33.124953 20183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:33.124961 20184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:33.124969 20186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:33.125197 20178 server_base.cc:1034] running on GCE node
I20260807 05:08:33.125728 20178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:33.126322 20178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:33.127506 20178 hybrid_clock.cc:648] HybridClock initialized: now 1786079313127492 us; error 23 us; skew 500 ppm
Aug 07 05:08:33 dist-test-slave-1x4g krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079313, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:33.130565 20178 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:08:33.131445 20178 webserver.cc:460] Webserver started at http://127.1.17.194:36243/ using document root <none> and password file <none>
I20260807 05:08:33.132054 20178 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:33.132148 20178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:33.132390 20178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:33.134354 20178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "252dc827f8b2446db01a9c6e12799dbd"
format_stamp: "Formatted at 2026-08-07 05:08:33 on dist-test-slave-1x4g"
server_key: "123286cdde546e4cf1eef8c5e0402c53"
server_key_iv: "5415e778a11583f04bca0207968857e2"
server_key_version: "encryptionkey@0"
I20260807 05:08:33.134891 20178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "252dc827f8b2446db01a9c6e12799dbd"
format_stamp: "Formatted at 2026-08-07 05:08:33 on dist-test-slave-1x4g"
server_key: "123286cdde546e4cf1eef8c5e0402c53"
server_key_iv: "5415e778a11583f04bca0207968857e2"
server_key_version: "encryptionkey@0"
I20260807 05:08:33.138645 20178 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260807 05:08:33.140972 20193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:33.142091 20178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:08:33.142220 20178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "252dc827f8b2446db01a9c6e12799dbd"
format_stamp: "Formatted at 2026-08-07 05:08:33 on dist-test-slave-1x4g"
server_key: "123286cdde546e4cf1eef8c5e0402c53"
server_key_iv: "5415e778a11583f04bca0207968857e2"
server_key_version: "encryptionkey@0"
I20260807 05:08:33.142325 20178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:33.152343 20178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:33.154937 20178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:33.155134 20178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:33.155654 20178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:33.156424 20178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:33.156493 20178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:33.156562 20178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:33.156618 20178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:33.165130 20178 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:35495
I20260807 05:08:33.165155 20311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:35495 every 8 connection(s)
I20260807 05:08:33.165915 20178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:08:33.167986 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 20178
I20260807 05:08:33.168125 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:08:33.168529 1095 external_mini_cluster.cc:1428] Setting key 3818ace7f47e4466dbc4d2efca6a0679
Aug 07 05:08:33 dist-test-slave-1x4g krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079313, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
I20260807 05:08:33.180356 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.167580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:57085 (local address 127.1.17.254:43611)
0807 05:08:33.167710 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.167714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.168621 (+ 907us) server_negotiation.cc:407] Connection header received
0807 05:08:33.169777 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.169780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.169822 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.169910 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:33.171258 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.171868 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.173667 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.173888 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.173973 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.176766 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:33.176770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:33.176800 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:33.176828 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:33.178289 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:33.178792 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:33.178796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:33.178798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:33.178879 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:33.179347 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:33.179350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:33.179352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:33.179571 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:33.179691 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.180145 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.180237 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 05:08:33.181223 20314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.167988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43611 (local address 127.1.17.194:57085)
0807 05:08:33.168481 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.168520 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.169559 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.170160 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.170169 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.170537 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:33.171116 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.171133 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.171999 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.172002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.173538 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.173551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.174047 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.174054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.174287 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.174883 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:33.174916 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:33.176579 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:33.178408 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:33.178413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:33.178427 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:33.178686 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:33.178991 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:33.178994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:33.178996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:33.179215 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:33.179698 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:33.179706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:33.179919 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.180081 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.180292 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":135,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
I20260807 05:08:33.182507 20312 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43611
I20260807 05:08:33.182713 20312 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:33.183205 20312 heartbeater.cc:511] Master 127.1.17.254:43611 requested a full tablet report, sending...
I20260807 05:08:33.184252 19971 ts_manager.cc:194] Registered new tserver with Master: 252dc827f8b2446db01a9c6e12799dbd (127.1.17.194:35495)
I20260807 05:08:33.184886 19971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:57085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:08:33.197790 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43611
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:33.317494 20317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:33.317786 20317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:33.317868 20317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:33.321820 20317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:33.321877 20317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:33.321983 20317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:08:33.327191 20317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:43611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:33.328258 20317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:33.329408 20317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:33.339510 20325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:33.339646 20317 server_base.cc:1034] running on GCE node
W20260807 05:08:33.339531 20322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:33.339562 20323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:33.340298 20317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:33.341025 20317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:33.342219 20317 hybrid_clock.cc:648] HybridClock initialized: now 1786079313342192 us; error 50 us; skew 500 ppm
Aug 07 05:08:33 dist-test-slave-1x4g krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079313, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:33.345263 20317 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:08:33.346239 20317 webserver.cc:460] Webserver started at http://127.1.17.195:36479/ using document root <none> and password file <none>
I20260807 05:08:33.346807 20317 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:33.346868 20317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:33.347078 20317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:33.349226 20317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "6f89d5b498994fcfab62249175922bb4"
format_stamp: "Formatted at 2026-08-07 05:08:33 on dist-test-slave-1x4g"
server_key: "7b3fbf74fb4a11038d7c0ae02fdf13ed"
server_key_iv: "bd49cd48841c576c7d6d57919126adc0"
server_key_version: "encryptionkey@0"
I20260807 05:08:33.349809 20317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "6f89d5b498994fcfab62249175922bb4"
format_stamp: "Formatted at 2026-08-07 05:08:33 on dist-test-slave-1x4g"
server_key: "7b3fbf74fb4a11038d7c0ae02fdf13ed"
server_key_iv: "bd49cd48841c576c7d6d57919126adc0"
server_key_version: "encryptionkey@0"
I20260807 05:08:33.353596 20317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 05:08:33.356005 20333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:33.357314 20317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:33.357442 20317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "6f89d5b498994fcfab62249175922bb4"
format_stamp: "Formatted at 2026-08-07 05:08:33 on dist-test-slave-1x4g"
server_key: "7b3fbf74fb4a11038d7c0ae02fdf13ed"
server_key_iv: "bd49cd48841c576c7d6d57919126adc0"
server_key_version: "encryptionkey@0"
I20260807 05:08:33.357546 20317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:33.375311 20317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:33.378782 20317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:33.378999 20317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:33.379629 20317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:33.380545 20317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:33.380594 20317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:33.380666 20317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:33.380709 20317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:33.389679 20317 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:40891
I20260807 05:08:33.389707 20452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:40891 every 8 connection(s)
I20260807 05:08:33.390601 20317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:08:33.394573 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 20317
I20260807 05:08:33.394726 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:08:33.395046 1095 external_mini_cluster.cc:1428] Setting key 5115955ed1603b29a75620ca05f539c7
Aug 07 05:08:33 dist-test-slave-1x4g krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079313, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:33.404160 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.392412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:52407 (local address 127.1.17.254:43611)
0807 05:08:33.392569 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.392574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.393542 (+ 968us) server_negotiation.cc:407] Connection header received
0807 05:08:33.394402 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.394405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.394466 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.394571 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:33.396001 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.396806 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.398106 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.398318 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.398399 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.400759 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:33.400770 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:33.400830 (+ 60us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:33.400870 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:33.402146 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:33.402660 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:33.402664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:33.402666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:33.402750 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:33.403101 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:33.403104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:33.403106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:33.403291 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:33.403401 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.403877 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.404001 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:08:33.405160 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.392907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:43611 (local address 127.1.17.195:52407)
0807 05:08:33.393366 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.393408 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.394171 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.394801 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.394810 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.395231 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:33.395826 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.395841 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.396933 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.396937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.397995 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.398005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.398452 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.398455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.398592 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.399137 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:33.399153 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:33.400597 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:33.402270 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:33.402276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:33.402291 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:33.402553 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:33.402844 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:33.402848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:33.402850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:33.402999 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:33.403400 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:33.403408 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:33.403643 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.403834 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.404157 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":107,"threads_started":1}
I20260807 05:08:33.406234 20453 heartbeater.cc:347] Connected to a master server at 127.1.17.254:43611
I20260807 05:08:33.406490 20453 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:33.406987 20453 heartbeater.cc:511] Master 127.1.17.254:43611 requested a full tablet report, sending...
I20260807 05:08:33.408114 19971 ts_manager.cc:194] Registered new tserver with Master: 6f89d5b498994fcfab62249175922bb4 (127.1.17.195:40891)
I20260807 05:08:33.408896 19971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:52407
I20260807 05:08:33.409305 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:08:33.418530 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.411318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.1.17.254:43611)
0807 05:08:33.411436 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.411442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.411592 (+ 150us) server_negotiation.cc:407] Connection header received
0807 05:08:33.411688 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.411690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.411722 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.411788 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:33.412389 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.412931 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.413744 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.413971 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.414047 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.415632 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:33.415635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:33.415637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:33.415658 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:33.417037 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:33.417482 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:33.417484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:33.417486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:33.417556 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:33.417819 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:33.417821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:33.417822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:33.418021 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:33.418134 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.418350 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.418425 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:08:33.421166 19971 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:08:33.424089 19971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:08:33.438752 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.433366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39786 (local address 127.1.17.194:35495)
0807 05:08:33.434479 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.434487 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.434506 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:08:33.434569 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.434573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.434757 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.434913 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:33.435088 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.435938 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.437512 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.438070 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.438198 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.438350 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.438531 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.438624 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015,"thread_start_us":139,"threads_started":1}
I20260807 05:08:33.438741 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.432612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:35495 (local address 127.0.0.1:39786)
0807 05:08:33.433030 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.433080 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.433211 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.434881 (+ 1670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.434884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.434901 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:33.435000 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.435006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.436138 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.436142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.437394 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.437402 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.438196 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.438200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.438313 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.438348 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.438476 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.438597 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":40,"thread_start_us":100,"threads_started":1}
I20260807 05:08:33.438979 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.432955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44631 (local address 127.0.0.1:37958)
0807 05:08:33.434115 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.434129 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.434223 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.434674 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.434677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.434695 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:33.434777 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.434783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.436327 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.436332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.437400 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.437411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.438623 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.438626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.438703 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.438721 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.438785 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.438845 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"mutex_wait_us":26,"thread_start_us":82,"threads_started":1}
I20260807 05:08:33.439038 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.432372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.1.17.195:40891)
0807 05:08:33.432729 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.432742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.433354 (+ 612us) server_negotiation.cc:407] Connection header received
0807 05:08:33.433436 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.433441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.433656 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.433810 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:33.434310 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.435452 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.437269 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.438094 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.438205 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.438362 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.438808 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.438899 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
I20260807 05:08:33.439347 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.433765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37958 (local address 127.1.17.193:44631)
0807 05:08:33.434063 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.434081 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.434195 (+ 114us) server_negotiation.cc:407] Connection header received
0807 05:08:33.434334 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.434339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.434521 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.434677 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:33.435136 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.436177 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.437540 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.438331 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.438435 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.438561 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.439096 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.439191 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
I20260807 05:08:33.441010 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.432191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:40891 (local address 127.0.0.1:33996)
0807 05:08:33.433115 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.433131 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.433225 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.434012 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.434019 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.434044 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:33.434191 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.434199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.435615 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.435618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.437145 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.437156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.438441 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.438444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.438532 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.438553 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.438626 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.438691 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":53,"thread_start_us":78,"threads_started":1}
I20260807 05:08:33.442589 20104 tablet_service.cc:1467] Processing CreateTablet for tablet a612b14e122540899219b6b41f989bfe (DEFAULT_TABLE table=test-table [id=f212f46dbdfe4b8eaed77db06f560bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:33.442543 20242 tablet_service.cc:1467] Processing CreateTablet for tablet a612b14e122540899219b6b41f989bfe (DEFAULT_TABLE table=test-table [id=f212f46dbdfe4b8eaed77db06f560bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:33.443184 20383 tablet_service.cc:1467] Processing CreateTablet for tablet a612b14e122540899219b6b41f989bfe (DEFAULT_TABLE table=test-table [id=f212f46dbdfe4b8eaed77db06f560bbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:33.443708 20104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a612b14e122540899219b6b41f989bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:33.443709 20242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a612b14e122540899219b6b41f989bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:33.444121 20383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a612b14e122540899219b6b41f989bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:33.450115 20470 tablet_bootstrap.cc:492] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Bootstrap starting.
I20260807 05:08:33.451951 20471 tablet_bootstrap.cc:492] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: Bootstrap starting.
I20260807 05:08:33.452497 20470 tablet_bootstrap.cc:654] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:33.453140 20472 tablet_bootstrap.cc:492] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Bootstrap starting.
I20260807 05:08:33.453688 20470 log.cc:824] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:33.454265 20471 tablet_bootstrap.cc:654] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:33.455466 20470 tablet_bootstrap.cc:492] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: No bootstrap required, opened a new log
I20260807 05:08:33.455516 20471 log.cc:824] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:33.455672 20472 tablet_bootstrap.cc:654] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:33.455704 20470 ts_tablet_manager.cc:1397] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:08:33.456456 20472 log.cc:824] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:33.457564 20471 tablet_bootstrap.cc:492] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: No bootstrap required, opened a new log
I20260807 05:08:33.457711 20471 ts_tablet_manager.cc:1397] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 05:08:33.459105 20470 raft_consensus.cc:348] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.459362 20470 raft_consensus.cc:374] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:33.459421 20470 raft_consensus.cc:729] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f89d5b498994fcfab62249175922bb4, State: Initialized, Role: FOLLOWER
I20260807 05:08:33.459851 20470 consensus_queue.cc:260] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.460530 20453 heartbeater.cc:503] Master 127.1.17.254:43611 was elected leader, sending a full tablet report...
I20260807 05:08:33.460620 20471 raft_consensus.cc:348] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.460852 20471 raft_consensus.cc:374] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:33.460903 20471 raft_consensus.cc:729] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259f4b44c4634262b7bbc25e3b52b3b5, State: Initialized, Role: FOLLOWER
I20260807 05:08:33.460866 20470 ts_tablet_manager.cc:1428] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:08:33.461378 20471 consensus_queue.cc:260] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.462061 20173 heartbeater.cc:503] Master 127.1.17.254:43611 was elected leader, sending a full tablet report...
I20260807 05:08:33.462339 20471 ts_tablet_manager.cc:1428] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:08:33.462463 20472 tablet_bootstrap.cc:492] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: No bootstrap required, opened a new log
I20260807 05:08:33.462647 20472 ts_tablet_manager.cc:1397] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260807 05:08:33.466534 20472 raft_consensus.cc:348] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.466814 20472 raft_consensus.cc:374] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:33.466888 20472 raft_consensus.cc:729] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252dc827f8b2446db01a9c6e12799dbd, State: Initialized, Role: FOLLOWER
I20260807 05:08:33.467317 20472 consensus_queue.cc:260] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.467887 20312 heartbeater.cc:503] Master 127.1.17.254:43611 was elected leader, sending a full tablet report...
I20260807 05:08:33.468151 20472 ts_tablet_manager.cc:1428] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 05:08:33.641588 20454 tablet.cc:2369] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:08:33.667083 20313 tablet.cc:2369] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 05:08:33.710649 20174 tablet.cc:2369] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:08:33.867229 20476 raft_consensus.cc:482] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:33.867492 20476 raft_consensus.cc:504] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.868690 20476 leader_election.cc:288] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 259f4b44c4634262b7bbc25e3b52b3b5 (127.1.17.193:44631), 252dc827f8b2446db01a9c6e12799dbd (127.1.17.194:35495)
I20260807 05:08:33.871692 20478 raft_consensus.cc:482] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:33.871871 20478 raft_consensus.cc:504] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.872711 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.869093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:55621 (local address 127.1.17.193:44631)
0807 05:08:33.869232 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.869247 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.869264 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:08:33.869333 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.869335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.869386 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.869495 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:33.869839 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.870379 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.871560 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.872175 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.872298 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.872348 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.872530 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.872598 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:08:33.872712 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.868984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44631 (local address 127.1.17.195:55621)
0807 05:08:33.869096 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.869110 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.869231 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.869570 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.869573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.869594 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:33.869719 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.869728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.870489 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.870492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.871362 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.871372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.872291 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.872295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.872399 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.872420 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.872494 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.872561 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":1}
I20260807 05:08:33.872857 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.869092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:57463 (local address 127.1.17.194:35495)
0807 05:08:33.869231 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.869247 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.869494 (+ 247us) server_negotiation.cc:407] Connection header received
0807 05:08:33.869608 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.869611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.869659 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.869741 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:33.870027 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.870552 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.871696 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.872260 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.872365 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.872418 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.872648 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.872727 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 05:08:33.873208 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.868984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:35495 (local address 127.1.17.195:57463)
0807 05:08:33.869380 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.869394 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.869503 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.869758 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.869760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.869791 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:33.869923 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.869929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.870689 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.870692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.871517 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.871529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.872423 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.872426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.872522 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.872539 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.872986 (+ 447us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.873086 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":131,"thread_start_us":97,"threads_started":1}
I20260807 05:08:33.873256 20478 leader_election.cc:288] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f89d5b498994fcfab62249175922bb4 (127.1.17.195:40891), 259f4b44c4634262b7bbc25e3b52b3b5 (127.1.17.193:44631)
I20260807 05:08:33.873469 20125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a612b14e122540899219b6b41f989bfe" candidate_uuid: "6f89d5b498994fcfab62249175922bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" is_pre_election: true
I20260807 05:08:33.873766 20125 raft_consensus.cc:2454] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f89d5b498994fcfab62249175922bb4 in term 0.
I20260807 05:08:33.874181 20263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a612b14e122540899219b6b41f989bfe" candidate_uuid: "6f89d5b498994fcfab62249175922bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "252dc827f8b2446db01a9c6e12799dbd" is_pre_election: true
I20260807 05:08:33.874279 20335 leader_election.cc:302] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 259f4b44c4634262b7bbc25e3b52b3b5, 6f89d5b498994fcfab62249175922bb4; no voters:
I20260807 05:08:33.874392 20263 raft_consensus.cc:2454] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f89d5b498994fcfab62249175922bb4 in term 0.
I20260807 05:08:33.874575 20476 raft_consensus.cc:2781] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:08:33.874668 20476 raft_consensus.cc:482] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:08:33.874716 20476 raft_consensus.cc:3037] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:33.875993 20476 raft_consensus.cc:504] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.876385 20476 leader_election.cc:288] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [CANDIDATE]: Term 1 election: Requested vote from peers 259f4b44c4634262b7bbc25e3b52b3b5 (127.1.17.193:44631), 252dc827f8b2446db01a9c6e12799dbd (127.1.17.194:35495)
I20260807 05:08:33.876694 20125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a612b14e122540899219b6b41f989bfe" candidate_uuid: "6f89d5b498994fcfab62249175922bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "259f4b44c4634262b7bbc25e3b52b3b5"
I20260807 05:08:33.876817 20263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a612b14e122540899219b6b41f989bfe" candidate_uuid: "6f89d5b498994fcfab62249175922bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "252dc827f8b2446db01a9c6e12799dbd"
I20260807 05:08:33.876832 20125 raft_consensus.cc:3037] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:33.876931 20263 raft_consensus.cc:3037] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:33.877880 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.873546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:54389 (local address 127.1.17.195:40891)
0807 05:08:33.873655 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.873659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.873822 (+ 163us) server_negotiation.cc:407] Connection header received
0807 05:08:33.873995 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.873998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.874043 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.874093 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:33.874492 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.875106 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.876617 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.877247 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.877344 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.877385 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.877694 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.877775 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:08:33.878099 20125 raft_consensus.cc:2454] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f89d5b498994fcfab62249175922bb4 in term 1.
I20260807 05:08:33.878099 20263 raft_consensus.cc:2454] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f89d5b498994fcfab62249175922bb4 in term 1.
I20260807 05:08:33.878252 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.873478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:40891 (local address 127.1.17.194:54389)
0807 05:08:33.873743 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.873756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.873886 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.874220 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.874223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.874241 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:33.874354 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.874361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.875221 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.875238 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.876462 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.876472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.877476 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.877479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.877570 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.877589 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.878039 (+ 450us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.878118 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260807 05:08:33.878441 20337 leader_election.cc:302] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252dc827f8b2446db01a9c6e12799dbd, 6f89d5b498994fcfab62249175922bb4; no voters:
I20260807 05:08:33.878796 20476 raft_consensus.cc:2781] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:33.878748 20404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a612b14e122540899219b6b41f989bfe" candidate_uuid: "252dc827f8b2446db01a9c6e12799dbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f89d5b498994fcfab62249175922bb4" is_pre_election: true
I20260807 05:08:33.878949 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.874379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:47813 (local address 127.1.17.193:44631)
0807 05:08:33.875105 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.875108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.875118 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:08:33.875149 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.875151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.875209 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.875294 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:33.875580 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.876290 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.877812 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.878304 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.878441 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.878483 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.878788 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.878847 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653}
I20260807 05:08:33.878963 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.873815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44631 (local address 127.1.17.194:47813)
0807 05:08:33.874130 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.874142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:33.874232 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:33.875330 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:33.875333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:33.875350 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:33.875463 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.875469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.876432 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.876435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.877682 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:33.877689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.878576 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.878579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:33.878676 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.878700 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:08:33.878771 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 05:08:33.878832 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":22,"thread_start_us":87,"threads_started":1}
I20260807 05:08:33.878985 20476 raft_consensus.cc:686] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 1 LEADER]: Becoming Leader. State: Replica: 6f89d5b498994fcfab62249175922bb4, State: Running, Role: LEADER
I20260807 05:08:33.879295 20476 consensus_queue.cc:237] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } }
I20260807 05:08:33.879563 20125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a612b14e122540899219b6b41f989bfe" candidate_uuid: "252dc827f8b2446db01a9c6e12799dbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" is_pre_election: true
I20260807 05:08:33.879701 20125 raft_consensus.cc:2379] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 252dc827f8b2446db01a9c6e12799dbd in current term 1: Already voted for candidate 6f89d5b498994fcfab62249175922bb4 in this term.
I20260807 05:08:33.882113 20196 leader_election.cc:302] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 252dc827f8b2446db01a9c6e12799dbd; no voters: 259f4b44c4634262b7bbc25e3b52b3b5, 6f89d5b498994fcfab62249175922bb4
I20260807 05:08:33.882344 20478 raft_consensus.cc:2726] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 05:08:33.882788 19971 catalog_manager.cc:5495] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f89d5b498994fcfab62249175922bb4 (127.1.17.195). New cstate: current_term: 1 leader_uuid: "6f89d5b498994fcfab62249175922bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f89d5b498994fcfab62249175922bb4" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 40891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 } health_report { overall_health: UNKNOWN } } }
I20260807 05:08:33.897513 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.893830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.1.17.195:40891)
0807 05:08:33.893971 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:33.893975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:33.894065 (+ 90us) server_negotiation.cc:407] Connection header received
0807 05:08:33.894256 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:33.894259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:33.894304 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:33.894425 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:08:33.894669 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.895119 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.895966 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:33.896174 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:33.896276 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:33.896617 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:08:33.897069 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:08:33.897236 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:33.897329 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 05:08:33.897362 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 05:08:33.902840 20125 raft_consensus.cc:1261] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5 [term 1 FOLLOWER]: Refusing update from remote peer 6f89d5b498994fcfab62249175922bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:33.902882 20263 raft_consensus.cc:1261] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 1 FOLLOWER]: Refusing update from remote peer 6f89d5b498994fcfab62249175922bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:33.903481 20476 consensus_queue.cc:1035] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "252dc827f8b2446db01a9c6e12799dbd" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 35495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:33.903638 20482 consensus_queue.cc:1035] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "259f4b44c4634262b7bbc25e3b52b3b5" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 44631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:33.911629 20487 mvcc.cc:205] Tried to move back new op lower bound from 7315780869739786240 to 7315780869652193280. Current Snapshot: MvccSnapshot[applied={T|T < 7315780869739786240}]
I20260807 05:08:33.912876 20486 mvcc.cc:205] Tried to move back new op lower bound from 7315780869739786240 to 7315780869652193280. Current Snapshot: MvccSnapshot[applied={T|T < 7315780869739786240}]
I20260807 05:08:33.914111 20485 mvcc.cc:205] Tried to move back new op lower bound from 7315780869739786240 to 7315780869652193280. Current Snapshot: MvccSnapshot[applied={T|T < 7315780869739786240 or (T in {7315780869739786240})}]
I20260807 05:08:33.919687 19971 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:33.919850 19971 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:33.922236 19971 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f40bfba33fdd4cf783f5080c38300194: Sending DeleteTablet for 3 replicas of tablet a612b14e122540899219b6b41f989bfe
I20260807 05:08:33.923044 20242 tablet_service.cc:1514] Processing DeleteTablet for tablet a612b14e122540899219b6b41f989bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:33 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:39786
I20260807 05:08:33.923044 20104 tablet_service.cc:1514] Processing DeleteTablet for tablet a612b14e122540899219b6b41f989bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:33 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:37958
I20260807 05:08:33.923022 20383 tablet_service.cc:1514] Processing DeleteTablet for tablet a612b14e122540899219b6b41f989bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:33 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:33996
I20260807 05:08:33.923532 20495 tablet_replica.cc:330] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: stopping tablet replica
I20260807 05:08:33.923862 20496 tablet_replica.cc:330] T a612b14e122540899219b6b41f989bfe P 259f4b44c4634262b7bbc25e3b52b3b5: stopping tablet replica
I20260807 05:08:33.923873 20495 raft_consensus.cc:2229] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:33.924158 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 20037
I20260807 05:08:33.924165 20495 raft_consensus.cc:2258] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:33.926783 20495 ts_tablet_manager.cc:1905] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:33.929613 20495 ts_tablet_manager.cc:1918] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:33.929718 20495 log.cc:1196] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/a612b14e122540899219b6b41f989bfe
I20260807 05:08:33.930354 20495 ts_tablet_manager.cc:1939] T a612b14e122540899219b6b41f989bfe P 252dc827f8b2446db01a9c6e12799dbd: Deleting consensus metadata
I20260807 05:08:33.931578 20497 tablet_replica.cc:330] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: stopping tablet replica
I20260807 05:08:33.931573 19958 catalog_manager.cc:4841] TS 252dc827f8b2446db01a9c6e12799dbd (127.1.17.194:35495): tablet a612b14e122540899219b6b41f989bfe (table test-table [id=f212f46dbdfe4b8eaed77db06f560bbc]) successfully deleted
I20260807 05:08:33.931849 20497 raft_consensus.cc:2229] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:08:33.932212 20497 raft_consensus.cc:2258] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:33.933465 20497 ts_tablet_manager.cc:1905] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 05:08:33.934422 19956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:44631 (error 108)
I20260807 05:08:33.935359 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:33.935060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:44631 (local address 127.0.0.1:37960)
0807 05:08:33.935187 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:33.935263 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:44631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 05:08:33.935496 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 20178
I20260807 05:08:33.937340 20497 ts_tablet_manager.cc:1918] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:33.937433 20497 log.cc:1196] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/a612b14e122540899219b6b41f989bfe
I20260807 05:08:33.937744 20497 ts_tablet_manager.cc:1939] T a612b14e122540899219b6b41f989bfe P 6f89d5b498994fcfab62249175922bb4: Deleting consensus metadata
W20260807 05:08:33.937847 19956 catalog_manager.cc:4584] TS 259f4b44c4634262b7bbc25e3b52b3b5 (127.1.17.193:44631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a612b14e122540899219b6b41f989bfe: Network error: Client connection negotiation failed: client connection to 127.1.17.193:44631: connect: Connection refused (error 111)
I20260807 05:08:33.938836 19956 catalog_manager.cc:4841] TS 6f89d5b498994fcfab62249175922bb4 (127.1.17.195:40891): tablet a612b14e122540899219b6b41f989bfe (table test-table [id=f212f46dbdfe4b8eaed77db06f560bbc]) successfully deleted
I20260807 05:08:33.943684 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 20317
I20260807 05:08:33.949537 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 19939
2026-08-07T05:08:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2634 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T05:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:33Z Disabled control of system clock
I20260807 05:08:33.981746 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:37033
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:37033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:08:34.093597 20502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:34.093919 20502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:34.093986 20502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:34.097900 20502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:34.097952 20502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:34.097970 20502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:34.098016 20502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:34.098049 20502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:34.102710 20502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:37033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:37033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:34.103645 20502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:34.105027 20502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:34.113294 20512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:34.113344 20509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:34.113344 20508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:34.113577 20502 server_base.cc:1034] running on GCE node
I20260807 05:08:34.114151 20502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:34.115152 20502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:34.116309 20502 hybrid_clock.cc:648] HybridClock initialized: now 1786079314116284 us; error 38 us; skew 500 ppm
I20260807 05:08:34.121474 20502 webserver.cc:460] Webserver started at http://127.1.17.254:37345/ using document root <none> and password file <none>
I20260807 05:08:34.122037 20502 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:34.122097 20502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:34.122335 20502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:34.124346 20502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "44aaba0a66ab479e83b6ecdad26b52fd"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "89562c2e4918ff8ead72387dc8f2b26f"
server_key_iv: "266ab4095221f5e7271a718db82e1f38"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.124975 20502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "44aaba0a66ab479e83b6ecdad26b52fd"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "89562c2e4918ff8ead72387dc8f2b26f"
server_key_iv: "266ab4095221f5e7271a718db82e1f38"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.128525 20502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:08:34.130836 20518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.131971 20502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:34.132119 20502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "44aaba0a66ab479e83b6ecdad26b52fd"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "89562c2e4918ff8ead72387dc8f2b26f"
server_key_iv: "266ab4095221f5e7271a718db82e1f38"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.132226 20502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:34.148532 20502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:34.149230 20502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:34.149436 20502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:34.156273 20502 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:37033
I20260807 05:08:34.156278 20574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:37033 every 8 connection(s)
I20260807 05:08:34.157286 20502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:34.157879 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 20502
I20260807 05:08:34.158010 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:34.158326 1095 external_mini_cluster.cc:1428] Setting key a37c06046332d5a487581257e2d89845
I20260807 05:08:34.161077 20575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:34.166602 20575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd: Bootstrap starting.
I20260807 05:08:34.168033 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.159784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.1.17.254:37033)
0807 05:08:34.160597 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.160626 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.160672 (+ 46us) server_negotiation.cc:407] Connection header received
0807 05:08:34.161666 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.161685 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.162067 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.162394 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:34.162832 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.163785 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.164670 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.165200 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.165378 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.166333 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:34.166338 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:34.166350 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:34.166767 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:34.166823 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:34.166842 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:34.166948 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.167120 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.167378 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":169,"threads_started":1}
I20260807 05:08:34.168886 20575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:34.169622 20575 log.cc:824] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:34.171536 20575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd: No bootstrap required, opened a new log
I20260807 05:08:34.175086 20575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37033 } }
I20260807 05:08:34.175330 20575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:34.175426 20575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44aaba0a66ab479e83b6ecdad26b52fd, State: Initialized, Role: FOLLOWER
I20260807 05:08:34.175836 20575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37033 } }
I20260807 05:08:34.175990 20575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:34.176075 20575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:34.176191 20575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:34.177129 20575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37033 } }
I20260807 05:08:34.177496 20575 leader_election.cc:302] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44aaba0a66ab479e83b6ecdad26b52fd; no voters:
I20260807 05:08:34.177776 20575 leader_election.cc:288] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:34.177906 20581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:34.178133 20581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [term 1 LEADER]: Becoming Leader. State: Replica: 44aaba0a66ab479e83b6ecdad26b52fd, State: Running, Role: LEADER
I20260807 05:08:34.178426 20581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37033 } }
I20260807 05:08:34.179124 20575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:34.180102 20582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37033 } } }
I20260807 05:08:34.180131 20583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44aaba0a66ab479e83b6ecdad26b52fd. Latest consensus state: current_term: 1 leader_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aaba0a66ab479e83b6ecdad26b52fd" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37033 } } }
I20260807 05:08:34.180256 20583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:34.180469 20582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:34.180621 20590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:34.183373 20590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:34.188517 20590 catalog_manager.cc:1347] Generated new cluster ID: 1ac4dd1ad063426eb9364c9e3d96aea6
I20260807 05:08:34.188578 20590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:34.203015 20590 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:34.204149 20590 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:34.211712 20590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd: Generated new TSK 0
I20260807 05:08:34.212507 20590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:34.215764 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:37033
--builtin_ntp_servers=127.1.17.212:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:08:34.329792 20601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:34.330015 20601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:34.330050 20601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:34.334003 20601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:34.334045 20601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:34.334096 20601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:08:34.338927 20601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:34.339906 20601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:34.341240 20601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:34.350615 20606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:34.350863 20601 server_base.cc:1034] running on GCE node
W20260807 05:08:34.350661 20609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:34.350636 20607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:34.351383 20601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:34.351997 20601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:34.353197 20601 hybrid_clock.cc:648] HybridClock initialized: now 1786079314353168 us; error 38 us; skew 500 ppm
I20260807 05:08:34.355087 20601 webserver.cc:460] Webserver started at http://127.1.17.193:41319/ using document root <none> and password file <none>
I20260807 05:08:34.355669 20601 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:34.355782 20601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:34.356029 20601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:34.358119 20601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "e0306e3a9ddc4def8e4927ad1660e6a1"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "b0cee8d4b57501e10bfc0054a7c150de"
server_key_iv: "2c72c6d84f7706458ec499f51ed1e73a"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.358711 20601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "e0306e3a9ddc4def8e4927ad1660e6a1"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "b0cee8d4b57501e10bfc0054a7c150de"
server_key_iv: "2c72c6d84f7706458ec499f51ed1e73a"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.362429 20601 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:08:34.364913 20615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.366029 20601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:34.366164 20601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "e0306e3a9ddc4def8e4927ad1660e6a1"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "b0cee8d4b57501e10bfc0054a7c150de"
server_key_iv: "2c72c6d84f7706458ec499f51ed1e73a"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.366272 20601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:34.376845 20601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:34.377431 20601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:34.377631 20601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:34.378176 20601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:34.379122 20601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:34.379190 20601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.379251 20601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:34.379320 20601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.388458 20601 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:38645
I20260807 05:08:34.388579 20734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:38645 every 8 connection(s)
I20260807 05:08:34.389418 20601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:08:34.394887 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 20601
I20260807 05:08:34.395037 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:08:34.395308 1095 external_mini_cluster.cc:1428] Setting key 9ae4c2fe9f5f2bcb21d62a7e8deb7af4
I20260807 05:08:34.397258 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:37033
--builtin_ntp_servers=127.1.17.212:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:08:34.399075 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.391275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:38085 (local address 127.1.17.254:37033)
0807 05:08:34.391532 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.391536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.392460 (+ 924us) server_negotiation.cc:407] Connection header received
0807 05:08:34.393902 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.393906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.393970 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.394092 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:34.394959 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.395630 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.397087 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.397320 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.397450 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.398572 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:34.398576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:34.398578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:34.398590 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:34.398616 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:34.398624 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:34.398702 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.398857 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.398940 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260807 05:08:34.399812 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.391639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:37033 (local address 127.1.17.193:38085)
0807 05:08:34.392300 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.392338 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.393687 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.394266 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.394272 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.394353 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:34.394805 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.394818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.395760 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.395768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.396958 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.396968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.397533 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.397539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.397697 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.398320 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:34.398322 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:34.398360 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:34.398422 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:34.398704 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:34.398716 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:34.398725 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.398856 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.399122 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":111,"threads_started":1}
I20260807 05:08:34.401064 20735 heartbeater.cc:347] Connected to a master server at 127.1.17.254:37033
I20260807 05:08:34.401299 20735 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:34.401868 20735 heartbeater.cc:511] Master 127.1.17.254:37033 requested a full tablet report, sending...
I20260807 05:08:34.403400 20535 ts_manager.cc:194] Registered new tserver with Master: e0306e3a9ddc4def8e4927ad1660e6a1 (127.1.17.193:38645)
I20260807 05:08:34.404541 20535 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.193:38085
W20260807 05:08:34.508543 20738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:34.508872 20738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:34.508922 20738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:34.512980 20738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:34.513031 20738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:34.513080 20738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:08:34.517656 20738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:34.518643 20738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:34.519927 20738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:34.529654 20744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:34.529686 20747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:34.529866 20745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:34.529888 20738 server_base.cc:1034] running on GCE node
I20260807 05:08:34.530508 20738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:34.531097 20738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:34.532258 20738 hybrid_clock.cc:648] HybridClock initialized: now 1786079314532224 us; error 43 us; skew 500 ppm
I20260807 05:08:34.534106 20738 webserver.cc:460] Webserver started at http://127.1.17.194:36321/ using document root <none> and password file <none>
I20260807 05:08:34.534708 20738 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:34.534770 20738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:34.535019 20738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:34.537344 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "5f483ffd5d214e299d95f475f954928a"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "1d78b77dc7c48e81bc851d44fd091798"
server_key_iv: "2ff0fb1c1e79f26337a9a0e787b1ebef"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.537925 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "5f483ffd5d214e299d95f475f954928a"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "1d78b77dc7c48e81bc851d44fd091798"
server_key_iv: "2ff0fb1c1e79f26337a9a0e787b1ebef"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.541597 20738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:34.543941 20753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.545202 20738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:08:34.545336 20738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "5f483ffd5d214e299d95f475f954928a"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "1d78b77dc7c48e81bc851d44fd091798"
server_key_iv: "2ff0fb1c1e79f26337a9a0e787b1ebef"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.545450 20738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:34.570752 20738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:34.571480 20738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:34.571687 20738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:34.572283 20738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:34.573279 20738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:34.573352 20738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.573432 20738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:34.573486 20738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.582998 20738 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:41005
I20260807 05:08:34.583026 20872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:41005 every 8 connection(s)
I20260807 05:08:34.583869 20738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:08:34.585011 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 20738
I20260807 05:08:34.585137 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:08:34.585460 1095 external_mini_cluster.cc:1428] Setting key 37529d57edeea4ab96af376ed7233db2
I20260807 05:08:34.587735 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:37033
--builtin_ntp_servers=127.1.17.212:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:08:34.595129 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.585967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:35839 (local address 127.1.17.254:37033)
0807 05:08:34.586131 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.586136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.587554 (+ 1418us) server_negotiation.cc:407] Connection header received
0807 05:08:34.589752 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.589756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.589821 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.589965 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:34.590837 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.591663 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.592873 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.593312 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.593442 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.594495 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:34.594500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:34.594505 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:34.594523 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:34.594552 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:34.594566 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:34.594672 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.594828 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.594912 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:08:34.595855 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.586480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:37033 (local address 127.1.17.194:35839)
0807 05:08:34.587367 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.587463 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.589470 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.590122 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.590128 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.590213 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:34.590693 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.590706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.591800 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.591803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.592684 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.592718 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.593479 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.593482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.593631 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.594239 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:34.594241 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:34.594280 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:34.594341 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:34.594677 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:34.594684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:34.594688 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.594790 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.595002 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":159,"threads_started":1}
I20260807 05:08:34.597040 20873 heartbeater.cc:347] Connected to a master server at 127.1.17.254:37033
I20260807 05:08:34.597292 20873 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:34.597790 20873 heartbeater.cc:511] Master 127.1.17.254:37033 requested a full tablet report, sending...
I20260807 05:08:34.598875 20535 ts_manager.cc:194] Registered new tserver with Master: 5f483ffd5d214e299d95f475f954928a (127.1.17.194:41005)
I20260807 05:08:34.599732 20535 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.194:35839
W20260807 05:08:34.702418 20877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:34.702651 20877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:34.702687 20877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:34.706521 20877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:34.706564 20877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:34.706614 20877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:08:34.711359 20877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:37033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:34.712348 20877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:34.713833 20877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:34.723076 20882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:34.723109 20885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:34.723088 20883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:34.723232 20877 server_base.cc:1034] running on GCE node
I20260807 05:08:34.723762 20877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:34.724282 20877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:34.725440 20877 hybrid_clock.cc:648] HybridClock initialized: now 1786079314725406 us; error 43 us; skew 500 ppm
I20260807 05:08:34.727283 20877 webserver.cc:460] Webserver started at http://127.1.17.195:39165/ using document root <none> and password file <none>
I20260807 05:08:34.727823 20877 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:34.727880 20877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:34.728083 20877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:34.730062 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "b91ce7c717b24f63a5686525f6ddf858"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "4cc65baeaff1456447d73c474e091454"
server_key_iv: "6a471327f5edee6a08ab6e5c33617b09"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.730613 20877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "b91ce7c717b24f63a5686525f6ddf858"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "4cc65baeaff1456447d73c474e091454"
server_key_iv: "6a471327f5edee6a08ab6e5c33617b09"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.734082 20877 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:34.736492 20891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.737833 20877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:34.737967 20877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "b91ce7c717b24f63a5686525f6ddf858"
format_stamp: "Formatted at 2026-08-07 05:08:34 on dist-test-slave-1x4g"
server_key: "4cc65baeaff1456447d73c474e091454"
server_key_iv: "6a471327f5edee6a08ab6e5c33617b09"
server_key_version: "encryptionkey@0"
I20260807 05:08:34.738075 20877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:34.751657 20877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:34.752233 20877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:34.752415 20877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:34.752978 20877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:34.753846 20877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:34.753916 20877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.753975 20877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:34.754019 20877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:34.762573 20877 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:35101
I20260807 05:08:34.762602 21013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:35101 every 8 connection(s)
I20260807 05:08:34.763427 20877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:08:34.765329 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 20877
I20260807 05:08:34.765450 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:08:34.765720 1095 external_mini_cluster.cc:1428] Setting key 66ec718485db6f4e6dfd166d64233e7e
I20260807 05:08:34.772874 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.765316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:36963 (local address 127.1.17.254:37033)
0807 05:08:34.765489 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.765496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.766624 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 05:08:34.768131 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.768133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.768176 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.768239 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:34.769085 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.769658 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.770809 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.771044 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.771121 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.772282 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:34.772288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:34.772292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:34.772309 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:34.772335 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:34.772348 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:34.772450 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.772616 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.772685 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:08:34.773705 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.765727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:37033 (local address 127.1.17.195:36963)
0807 05:08:34.766473 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.766525 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.767955 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.768401 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.768406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.768487 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:34.768971 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.768984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.769763 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.769766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.770710 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.770719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.771173 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.771176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.771307 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.771964 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:34.771966 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:34.772004 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:34.772101 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:34.772454 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:34.772465 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:34.772473 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.772610 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.772988 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":124,"threads_started":1}
I20260807 05:08:34.774755 21014 heartbeater.cc:347] Connected to a master server at 127.1.17.254:37033
I20260807 05:08:34.774977 21014 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:34.775452 21014 heartbeater.cc:511] Master 127.1.17.254:37033 requested a full tablet report, sending...
I20260807 05:08:34.776391 20535 ts_manager.cc:194] Registered new tserver with Master: b91ce7c717b24f63a5686525f6ddf858 (127.1.17.195:35101)
I20260807 05:08:34.776988 20535 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.195:36963
I20260807 05:08:34.779477 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:08:34.785028 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.781581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.1.17.254:37033)
0807 05:08:34.781694 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.781698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.781857 (+ 159us) server_negotiation.cc:407] Connection header received
0807 05:08:34.781929 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.781931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.781968 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.782031 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:34.782284 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.782762 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.783587 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.783785 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.783861 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.784582 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:34.784584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:34.784586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:34.784597 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:34.784613 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:34.784620 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:34.784718 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.784906 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.784935 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:08:34.787590 20535 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:08:34.790613 20535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:08:34.804533 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.798672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59514 (local address 127.1.17.193:38645)
0807 05:08:34.799077 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.799089 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.799577 (+ 488us) server_negotiation.cc:407] Connection header received
0807 05:08:34.799664 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.799670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.799878 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.800048 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:34.800553 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.801793 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.803295 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.803879 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.804000 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.804185 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.804284 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.804376 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":103,"threads_started":1}
I20260807 05:08:34.804522 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.798537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38645 (local address 127.0.0.1:59514)
0807 05:08:34.799224 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.799271 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.799418 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.800240 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.800246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.800268 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:34.800419 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.800426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.801911 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.801914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.803178 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.803183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.804003 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.804007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.804109 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.804145 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.804257 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.804376 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":98,"thread_start_us":53,"threads_started":1}
I20260807 05:08:34.804819 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.798887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:35101 (local address 127.0.0.1:39896)
0807 05:08:34.799670 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.799681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.799762 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.800530 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.800533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.800551 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:34.800641 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.800651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.801626 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.801632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.803344 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.803354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.804304 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.804308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.804393 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.804414 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.804465 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.804517 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":293,"thread_start_us":113,"threads_started":1}
I20260807 05:08:34.805128 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.798646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:41005 (local address 127.0.0.1:58292)
0807 05:08:34.799381 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.799396 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.799496 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.800399 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.800402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.800417 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:34.800527 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.800534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.802036 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.802042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.803172 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.803183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.804219 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.804225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.804323 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.804347 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.804393 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.804463 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":105,"thread_start_us":86,"threads_started":1}
I20260807 05:08:34.805145 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.799266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.1.17.195:35101)
0807 05:08:34.800024 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.800037 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.800060 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:08:34.800141 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.800146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.800357 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.800551 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:34.800769 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.801514 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.803473 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.804194 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.804623 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.804839 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.804885 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.804974 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":193,"threads_started":1}
I20260807 05:08:34.805851 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.799293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.1.17.194:41005)
0807 05:08:34.799918 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.799936 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.799962 (+ 26us) server_negotiation.cc:407] Connection header received
0807 05:08:34.800043 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.800048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.800245 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.800672 (+ 427us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:34.800831 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.801929 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.803317 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.804076 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.805392 (+ 1316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.805561 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.805606 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.805685 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":173,"threads_started":1}
I20260807 05:08:34.808024 20802 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6d4dcec235440f914ba80ceea63247 (DEFAULT_TABLE table=test-table [id=d64c8e274cdc43649ce937091a386421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:34.807925 20942 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6d4dcec235440f914ba80ceea63247 (DEFAULT_TABLE table=test-table [id=d64c8e274cdc43649ce937091a386421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:34.807925 20664 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6d4dcec235440f914ba80ceea63247 (DEFAULT_TABLE table=test-table [id=d64c8e274cdc43649ce937091a386421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:34.808992 20802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6d4dcec235440f914ba80ceea63247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:34.809015 20664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6d4dcec235440f914ba80ceea63247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:34.809378 20942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6d4dcec235440f914ba80ceea63247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:34.815105 21033 tablet_bootstrap.cc:492] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Bootstrap starting.
I20260807 05:08:34.816705 21034 tablet_bootstrap.cc:492] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Bootstrap starting.
I20260807 05:08:34.817109 21035 tablet_bootstrap.cc:492] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: Bootstrap starting.
I20260807 05:08:34.817492 21033 tablet_bootstrap.cc:654] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:34.818526 21033 log.cc:824] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:34.818871 21034 tablet_bootstrap.cc:654] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:34.819072 21035 tablet_bootstrap.cc:654] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:34.819741 21034 log.cc:824] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:34.819769 21035 log.cc:824] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:34.820637 21033 tablet_bootstrap.cc:492] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: No bootstrap required, opened a new log
I20260807 05:08:34.820870 21033 ts_tablet_manager.cc:1397] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 05:08:34.821714 21035 tablet_bootstrap.cc:492] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: No bootstrap required, opened a new log
I20260807 05:08:34.821900 21035 ts_tablet_manager.cc:1397] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:08:34.821973 21034 tablet_bootstrap.cc:492] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: No bootstrap required, opened a new log
I20260807 05:08:34.822144 21034 ts_tablet_manager.cc:1397] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 05:08:34.824510 21033 raft_consensus.cc:348] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.824745 21033 raft_consensus.cc:374] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:34.824818 21033 raft_consensus.cc:729] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f483ffd5d214e299d95f475f954928a, State: Initialized, Role: FOLLOWER
I20260807 05:08:34.825265 21033 consensus_queue.cc:260] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.825958 20873 heartbeater.cc:503] Master 127.1.17.254:37033 was elected leader, sending a full tablet report...
I20260807 05:08:34.826300 21033 ts_tablet_manager.cc:1428] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 05:08:34.826299 21035 raft_consensus.cc:348] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.826556 21035 raft_consensus.cc:374] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:34.826617 21035 raft_consensus.cc:729] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0306e3a9ddc4def8e4927ad1660e6a1, State: Initialized, Role: FOLLOWER
I20260807 05:08:34.827036 21035 consensus_queue.cc:260] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.826967 21034 raft_consensus.cc:348] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.827214 21034 raft_consensus.cc:374] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:34.827266 21034 raft_consensus.cc:729] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b91ce7c717b24f63a5686525f6ddf858, State: Initialized, Role: FOLLOWER
I20260807 05:08:34.827706 21034 consensus_queue.cc:260] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.827991 21035 ts_tablet_manager.cc:1428] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 05:08:34.828293 20735 heartbeater.cc:503] Master 127.1.17.254:37033 was elected leader, sending a full tablet report...
I20260807 05:08:34.828701 21034 ts_tablet_manager.cc:1428] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 05:08:34.829066 21014 heartbeater.cc:503] Master 127.1.17.254:37033 was elected leader, sending a full tablet report...
W20260807 05:08:34.834770 20874 tablet.cc:2369] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:08:34.889572 21041 raft_consensus.cc:482] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:34.889781 21041 raft_consensus.cc:504] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
W20260807 05:08:34.890549 20736 tablet.cc:2369] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:08:34.891083 21041 leader_election.cc:288] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0306e3a9ddc4def8e4927ad1660e6a1 (127.1.17.193:38645), 5f483ffd5d214e299d95f475f954928a (127.1.17.194:41005)
I20260807 05:08:34.894618 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.891349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:33963 (local address 127.1.17.193:38645)
0807 05:08:34.891481 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.891487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.891504 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:08:34.891668 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.891671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.891727 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.891824 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:34.892166 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.892641 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.893689 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.894046 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.894163 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.894231 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.894408 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.894497 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:08:34.894618 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.891254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38645 (local address 127.1.17.195:33963)
0807 05:08:34.891401 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.891418 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.891523 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.891874 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.891878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.891896 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:34.892021 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.892027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.892743 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.892749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.893568 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.893576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.894156 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.894159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.894288 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.894305 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.894382 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.894507 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":41}
I20260807 05:08:34.894783 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.891578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:37767 (local address 127.1.17.194:41005)
0807 05:08:34.891701 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.891706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.891812 (+ 106us) server_negotiation.cc:407] Connection header received
0807 05:08:34.891982 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.891985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.892027 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.892124 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:34.892394 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.892868 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.893866 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.894237 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.894371 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.894414 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.894617 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.894683 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:08:34.894805 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.891357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:41005 (local address 127.1.17.195:37767)
0807 05:08:34.891722 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.891734 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:34.891837 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:34.892136 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:34.892139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:34.892165 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:34.892276 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.892281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.892983 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.892986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.893745 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:34.893753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.894371 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.894374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:34.894501 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.894518 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 05:08:34.894599 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 05:08:34.894662 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":45,"thread_start_us":158,"threads_started":1}
I20260807 05:08:34.895287 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6d4dcec235440f914ba80ceea63247" candidate_uuid: "b91ce7c717b24f63a5686525f6ddf858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" is_pre_election: true
I20260807 05:08:34.895386 20825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6d4dcec235440f914ba80ceea63247" candidate_uuid: "b91ce7c717b24f63a5686525f6ddf858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f483ffd5d214e299d95f475f954928a" is_pre_election: true
I20260807 05:08:34.895629 20684 raft_consensus.cc:2454] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b91ce7c717b24f63a5686525f6ddf858 in term 0.
I20260807 05:08:34.895629 20825 raft_consensus.cc:2454] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b91ce7c717b24f63a5686525f6ddf858 in term 0.
I20260807 05:08:34.896085 20895 leader_election.cc:302] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b91ce7c717b24f63a5686525f6ddf858, e0306e3a9ddc4def8e4927ad1660e6a1; no voters:
I20260807 05:08:34.896338 21041 raft_consensus.cc:2781] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:08:34.896415 21041 raft_consensus.cc:482] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:08:34.896445 21041 raft_consensus.cc:3037] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:34.897754 21041 raft_consensus.cc:504] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.898118 21041 leader_election.cc:288] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [CANDIDATE]: Term 1 election: Requested vote from peers e0306e3a9ddc4def8e4927ad1660e6a1 (127.1.17.193:38645), 5f483ffd5d214e299d95f475f954928a (127.1.17.194:41005)
I20260807 05:08:34.898406 20825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6d4dcec235440f914ba80ceea63247" candidate_uuid: "b91ce7c717b24f63a5686525f6ddf858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f483ffd5d214e299d95f475f954928a"
I20260807 05:08:34.898406 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6d4dcec235440f914ba80ceea63247" candidate_uuid: "b91ce7c717b24f63a5686525f6ddf858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1"
I20260807 05:08:34.898526 20684 raft_consensus.cc:3037] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:34.898526 20825 raft_consensus.cc:3037] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:34.899719 20825 raft_consensus.cc:2454] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b91ce7c717b24f63a5686525f6ddf858 in term 1.
I20260807 05:08:34.899719 20684 raft_consensus.cc:2454] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b91ce7c717b24f63a5686525f6ddf858 in term 1.
I20260807 05:08:34.900012 20893 leader_election.cc:302] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f483ffd5d214e299d95f475f954928a, b91ce7c717b24f63a5686525f6ddf858; no voters:
I20260807 05:08:34.900159 21041 raft_consensus.cc:2781] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:34.900367 21041 raft_consensus.cc:686] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 1 LEADER]: Becoming Leader. State: Replica: b91ce7c717b24f63a5686525f6ddf858, State: Running, Role: LEADER
I20260807 05:08:34.900697 21041 consensus_queue.cc:237] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } }
I20260807 05:08:34.903956 20533 catalog_manager.cc:5495] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 reported cstate change: term changed from 0 to 1, leader changed from <none> to b91ce7c717b24f63a5686525f6ddf858 (127.1.17.195). New cstate: current_term: 1 leader_uuid: "b91ce7c717b24f63a5686525f6ddf858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b91ce7c717b24f63a5686525f6ddf858" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 35101 } health_report { overall_health: HEALTHY } } }
I20260807 05:08:34.930104 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.926222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39908 (local address 127.1.17.195:35101)
0807 05:08:34.926357 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:34.926363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:34.926382 (+ 19us) server_negotiation.cc:407] Connection header received
0807 05:08:34.926579 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:34.926581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:34.926637 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:34.926748 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:08:34.927047 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.927545 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.928446 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:34.928686 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:34.928835 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:34.929188 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:08:34.929598 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:08:34.929775 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:34.929897 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 05:08:34.929936 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:08:34.936048 20825 raft_consensus.cc:1261] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 1 FOLLOWER]: Refusing update from remote peer b91ce7c717b24f63a5686525f6ddf858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:34.936048 20684 raft_consensus.cc:1261] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 1 FOLLOWER]: Refusing update from remote peer b91ce7c717b24f63a5686525f6ddf858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:34.936889 21041 consensus_queue.cc:1035] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0306e3a9ddc4def8e4927ad1660e6a1" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:34.937160 21043 consensus_queue.cc:1035] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f483ffd5d214e299d95f475f954928a" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 41005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:34.944630 21048 mvcc.cc:205] Tried to move back new op lower bound from 7315780873970278400 to 7315780873836318720. Current Snapshot: MvccSnapshot[applied={T|T < 7315780873970278400}]
I20260807 05:08:34.952112 20533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:34.952288 20533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:34.954669 20533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44aaba0a66ab479e83b6ecdad26b52fd: Sending DeleteTablet for 3 replicas of tablet 5b6d4dcec235440f914ba80ceea63247
I20260807 05:08:34.955415 20664 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6d4dcec235440f914ba80ceea63247 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:34 UTC) from {username='slave'} at 127.0.0.1:59514
I20260807 05:08:34.955454 20802 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6d4dcec235440f914ba80ceea63247 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:34 UTC) from {username='slave'} at 127.0.0.1:58292
I20260807 05:08:34.955550 20942 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6d4dcec235440f914ba80ceea63247 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:34 UTC) from {username='slave'} at 127.0.0.1:39896
I20260807 05:08:34.956115 21057 tablet_replica.cc:330] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1: stopping tablet replica
I20260807 05:08:34.956419 21057 raft_consensus.cc:2229] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:34.956791 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 20601
I20260807 05:08:34.956779 21057 raft_consensus.cc:2258] T 5b6d4dcec235440f914ba80ceea63247 P e0306e3a9ddc4def8e4927ad1660e6a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:34.957309 21058 tablet_replica.cc:330] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: stopping tablet replica
I20260807 05:08:34.957597 21058 raft_consensus.cc:2229] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:08:34.957952 21058 raft_consensus.cc:2258] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:34.958307 21056 tablet_replica.cc:330] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: stopping tablet replica
I20260807 05:08:34.958606 21056 raft_consensus.cc:2229] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:34.958943 21056 raft_consensus.cc:2258] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:34.960532 21056 ts_tablet_manager.cc:1905] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:34.961679 21058 ts_tablet_manager.cc:1905] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:34.964141 21056 ts_tablet_manager.cc:1918] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:34.964255 21056 log.cc:1196] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/5b6d4dcec235440f914ba80ceea63247
I20260807 05:08:34.964464 21058 ts_tablet_manager.cc:1918] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:34.964614 21058 log.cc:1196] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/5b6d4dcec235440f914ba80ceea63247
I20260807 05:08:34.964613 21056 ts_tablet_manager.cc:1939] T 5b6d4dcec235440f914ba80ceea63247 P 5f483ffd5d214e299d95f475f954928a: Deleting consensus metadata
I20260807 05:08:34.965014 21058 ts_tablet_manager.cc:1939] T 5b6d4dcec235440f914ba80ceea63247 P b91ce7c717b24f63a5686525f6ddf858: Deleting consensus metadata
I20260807 05:08:34.966013 20520 catalog_manager.cc:4841] TS 5f483ffd5d214e299d95f475f954928a (127.1.17.194:41005): tablet 5b6d4dcec235440f914ba80ceea63247 (table test-table [id=d64c8e274cdc43649ce937091a386421]) successfully deleted
I20260807 05:08:34.966243 20521 catalog_manager.cc:4841] TS b91ce7c717b24f63a5686525f6ddf858 (127.1.17.195:35101): tablet 5b6d4dcec235440f914ba80ceea63247 (table test-table [id=d64c8e274cdc43649ce937091a386421]) successfully deleted
W20260807 05:08:34.968915 20522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:38645 (error 108)
I20260807 05:08:34.969825 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 20738
I20260807 05:08:34.970013 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:34.969704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:38645 (local address 127.0.0.1:59516)
0807 05:08:34.969830 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:34.969910 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 05:08:34.971252 20522 catalog_manager.cc:4584] TS e0306e3a9ddc4def8e4927ad1660e6a1 (127.1.17.193:38645): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b6d4dcec235440f914ba80ceea63247: Network error: Client connection negotiation failed: client connection to 127.1.17.193:38645: connect: Connection refused (error 111)
I20260807 05:08:34.978062 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 20877
I20260807 05:08:34.984980 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 20502
2026-08-07T05:08:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1034 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:35 dist-test-slave-1x4g krb5kdc[21062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:35 dist-test-slave-1x4g krb5kdc[21062](info): set up 2 sockets
Aug 07 05:08:35 dist-test-slave-1x4g krb5kdc[21062](info): commencing operation
krb5kdc: starting...
W20260807 05:08:36.037952 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:36 dist-test-slave-1x4g krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:08:36.164417 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:40505
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:40505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:36.273479 21075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:36.273748 21075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:36.273826 21075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:36.277786 21075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:36.277859 21075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:36.277897 21075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:36.277954 21075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:36.277992 21075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:36.283190 21075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:40505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:40505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:36.284199 21075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:36.285261 21075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:08:36.293579 21075 server_base.cc:1034] running on GCE node
W20260807 05:08:36.293602 21081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:36.293642 21083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:36.293805 21080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:36.294363 21075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:36.295312 21075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:36.296473 21075 hybrid_clock.cc:648] HybridClock initialized: now 1786079316296459 us; error 23 us; skew 500 ppm
Aug 07 05:08:36 dist-test-slave-1x4g krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:36.299218 21075 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:08:36.300086 21075 webserver.cc:460] Webserver started at http://127.1.17.254:34473/ using document root <none> and password file <none>
I20260807 05:08:36.300612 21075 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:36.300702 21075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:36.300973 21075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:36.302910 21075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "a577f16184de48a7bebbadfd4bcae3e0"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "bc373278a581738d8b97b3c7e87a3785"
server_key_iv: "6ba98c9f91bc931d0e7aa1a1b0dc0aff"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.303500 21075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "a577f16184de48a7bebbadfd4bcae3e0"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "bc373278a581738d8b97b3c7e87a3785"
server_key_iv: "6ba98c9f91bc931d0e7aa1a1b0dc0aff"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.307101 21075 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 05:08:36.309345 21092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:36.310581 21075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:36.310714 21075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "a577f16184de48a7bebbadfd4bcae3e0"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "bc373278a581738d8b97b3c7e87a3785"
server_key_iv: "6ba98c9f91bc931d0e7aa1a1b0dc0aff"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.310817 21075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:36.328190 21075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:36.330821 21075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:36.331017 21075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:36.337919 21075 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:40505
I20260807 05:08:36.337946 21146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:40505 every 8 connection(s)
I20260807 05:08:36.338843 21075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:36.340504 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 21075
I20260807 05:08:36.340646 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:36.340986 1095 external_mini_cluster.cc:1428] Setting key 961d18528fab59a7a1bd99edc2501daf
I20260807 05:08:36.341908 21147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:36.347435 21147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0: Bootstrap starting.
Aug 07 05:08:36 dist-test-slave-1x4g krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:36.349720 21147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:36.350509 21147 log.cc:824] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:36.352600 21147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0: No bootstrap required, opened a new log
I20260807 05:08:36.354586 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:36.342381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.1.17.254:40505)
0807 05:08:36.342806 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:36.342821 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:36.342853 (+ 32us) server_negotiation.cc:407] Connection header received
0807 05:08:36.343497 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:36.343512 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:36.343805 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:36.344130 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:36.345073 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.346002 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.346845 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.347322 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.347489 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:36.350075 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:36.350080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:36.350092 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:36.350117 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:36.351892 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:36.352388 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:36.352392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:36.352396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:36.352504 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:36.352816 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:36.352819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:36.352820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:36.353176 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:36.353312 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:36.353617 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 05:08:36.353895 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":141,"threads_started":1}
I20260807 05:08:36.356513 21147 raft_consensus.cc:348] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a577f16184de48a7bebbadfd4bcae3e0" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40505 } }
I20260807 05:08:36.356784 21147 raft_consensus.cc:374] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:36.356871 21147 raft_consensus.cc:729] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a577f16184de48a7bebbadfd4bcae3e0, State: Initialized, Role: FOLLOWER
I20260807 05:08:36.357281 21147 consensus_queue.cc:260] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a577f16184de48a7bebbadfd4bcae3e0" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40505 } }
I20260807 05:08:36.357432 21147 raft_consensus.cc:388] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:36.357522 21147 raft_consensus.cc:482] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:36.357640 21147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:36.358466 21147 raft_consensus.cc:504] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a577f16184de48a7bebbadfd4bcae3e0" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40505 } }
I20260807 05:08:36.358784 21147 leader_election.cc:302] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a577f16184de48a7bebbadfd4bcae3e0; no voters:
I20260807 05:08:36.359097 21147 leader_election.cc:288] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:36.359171 21152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:36.359639 21152 raft_consensus.cc:686] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [term 1 LEADER]: Becoming Leader. State: Replica: a577f16184de48a7bebbadfd4bcae3e0, State: Running, Role: LEADER
I20260807 05:08:36.359916 21152 consensus_queue.cc:237] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a577f16184de48a7bebbadfd4bcae3e0" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40505 } }
I20260807 05:08:36.360309 21147 sys_catalog.cc:564] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:36.361570 21153 sys_catalog.cc:455] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a577f16184de48a7bebbadfd4bcae3e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a577f16184de48a7bebbadfd4bcae3e0" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40505 } } }
I20260807 05:08:36.361711 21153 sys_catalog.cc:458] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:36.362033 21154 sys_catalog.cc:455] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a577f16184de48a7bebbadfd4bcae3e0. Latest consensus state: current_term: 1 leader_uuid: "a577f16184de48a7bebbadfd4bcae3e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a577f16184de48a7bebbadfd4bcae3e0" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 40505 } } }
I20260807 05:08:36.362128 21154 sys_catalog.cc:458] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:36.362265 21163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:36.365164 21163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:36.370456 21163 catalog_manager.cc:1347] Generated new cluster ID: c83309ca04e64b6690247f34d3682fdd
I20260807 05:08:36.370518 21163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:36.384104 21163 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:36.385231 21163 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:36.391822 21163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0: Generated new TSK 0
I20260807 05:08:36.392447 21163 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.17.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
Entry for principal kudu/127.1.17.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab.
I20260807 05:08:36.426095 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40505
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:36.538939 21176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:36.539213 21176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:36.539294 21176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:36.543249 21176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:36.543335 21176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:36.543444 21176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:08:36.548844 21176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.193.keytab
--principal=kudu/127.1.17.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:36.549826 21176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:36.550794 21176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:36.560392 21184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:36.560390 21181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:36.560390 21182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:36.560665 21176 server_base.cc:1034] running on GCE node
I20260807 05:08:36.561133 21176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:36.561750 21176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:36.562942 21176 hybrid_clock.cc:648] HybridClock initialized: now 1786079316562929 us; error 25 us; skew 500 ppm
Aug 07 05:08:36 dist-test-slave-1x4g krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:36.565675 21176 init.cc:377] Logged in from keytab as kudu/127.1.17.193@KRBTEST.COM (short username kudu)
I20260807 05:08:36.566519 21176 webserver.cc:460] Webserver started at http://127.1.17.193:45301/ using document root <none> and password file <none>
I20260807 05:08:36.567077 21176 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:36.567164 21176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:36.567399 21176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:36.569440 21176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "2e12a17bc3254833bb987225bd201d52"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "9e4fc24430d17824c1f7ebdd09c69e70"
server_key_iv: "602bd24e64cb9bc9f0ed3d205fd55ada"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.570005 21176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "2e12a17bc3254833bb987225bd201d52"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "9e4fc24430d17824c1f7ebdd09c69e70"
server_key_iv: "602bd24e64cb9bc9f0ed3d205fd55ada"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.573724 21176 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:36.576025 21191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:36.577188 21176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:08:36.577314 21176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "2e12a17bc3254833bb987225bd201d52"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "9e4fc24430d17824c1f7ebdd09c69e70"
server_key_iv: "602bd24e64cb9bc9f0ed3d205fd55ada"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.577425 21176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:36.595902 21176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:36.598841 21176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:36.599056 21176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:36.599647 21176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:36.600494 21176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:36.600565 21176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:36.600636 21176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:36.600675 21176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:36.609961 21176 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:39421
I20260807 05:08:36.609994 21307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:39421 every 8 connection(s)
I20260807 05:08:36.610882 21176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:08:36.612839 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 21176
I20260807 05:08:36.612979 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:08:36.613258 1095 external_mini_cluster.cc:1428] Setting key b465e86e1afb520eebddc1f723ecb45a
Aug 07 05:08:36 dist-test-slave-1x4g krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.193@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.194@KRBTEST.COM; defaulting to no policy
I20260807 05:08:36.625998 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:36.612693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:58313 (local address 127.1.17.254:40505)
0807 05:08:36.612877 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:36.612882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:36.614400 (+ 1518us) server_negotiation.cc:407] Connection header received
0807 05:08:36.615296 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:36.615302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:36.615361 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:36.615460 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:36.616692 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.617383 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.619118 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.619338 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.619426 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:36.622288 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:36.622292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:36.622295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:36.622327 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:36.623949 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:36.624450 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:36.624456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:36.624460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:36.624552 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:36.624936 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:36.624941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:36.624944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:36.625203 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:36.625328 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:36.625770 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 05:08:36.625876 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 05:08:36.626822 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:36.613151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:40505 (local address 127.1.17.193:58313)
0807 05:08:36.613638 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:36.613688 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:36.615092 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:36.615644 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:36.615650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:36.616040 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:36.616513 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:36.616526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.617533 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.617540 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:36.618988 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:36.619002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.619490 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.619496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:36.619712 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:36.620358 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:36.620383 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:36.622102 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:36.624069 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:36.624074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:36.624085 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:36.624333 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:36.624651 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:36.624654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:36.624655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:36.624835 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:36.625320 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:36.625324 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:36.625558 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 05:08:36.625703 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 05:08:36.625939 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":150,"threads_started":1}
I20260807 05:08:36.628043 21308 heartbeater.cc:347] Connected to a master server at 127.1.17.254:40505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.194@KRBTEST.COM" created.
I20260807 05:08:36.628280 21308 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:36.628942 21308 heartbeater.cc:511] Master 127.1.17.254:40505 requested a full tablet report, sending...
I20260807 05:08:36.630626 21110 ts_manager.cc:194] Registered new tserver with Master: 2e12a17bc3254833bb987225bd201d52 (127.1.17.193:39421)
I20260807 05:08:36.631807 21110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.193@KRBTEST.COM'} at 127.1.17.193:58313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
Entry for principal kudu/127.1.17.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab.
I20260807 05:08:36.644284 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40505
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:36.758962 21314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:36.759224 21314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:36.759292 21314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:36.763537 21314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:36.763613 21314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:36.763693 21314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:08:36.768863 21314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.194.keytab
--principal=kudu/127.1.17.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:36.769888 21314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:36.770952 21314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:36.780411 21320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:36.780436 21322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:36.780431 21319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:36.780809 21314 server_base.cc:1034] running on GCE node
I20260807 05:08:36.781291 21314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:36.781917 21314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:36.783108 21314 hybrid_clock.cc:648] HybridClock initialized: now 1786079316783092 us; error 31 us; skew 500 ppm
Aug 07 05:08:36 dist-test-slave-1x4g krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:36.785976 21314 init.cc:377] Logged in from keytab as kudu/127.1.17.194@KRBTEST.COM (short username kudu)
I20260807 05:08:36.786756 21314 webserver.cc:460] Webserver started at http://127.1.17.194:41999/ using document root <none> and password file <none>
I20260807 05:08:36.787252 21314 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:36.787310 21314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:36.787484 21314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:36.789618 21314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "8ab9e92919014daca78070546788398f"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "b264a1bf46e38512797e342524306c21"
server_key_iv: "88e6344fbf48ddb903317bfdf55e87a4"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.790124 21314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "8ab9e92919014daca78070546788398f"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "b264a1bf46e38512797e342524306c21"
server_key_iv: "88e6344fbf48ddb903317bfdf55e87a4"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.793733 21314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:36.796104 21329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:36.797343 21314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:36.797479 21314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "8ab9e92919014daca78070546788398f"
format_stamp: "Formatted at 2026-08-07 05:08:36 on dist-test-slave-1x4g"
server_key: "b264a1bf46e38512797e342524306c21"
server_key_iv: "88e6344fbf48ddb903317bfdf55e87a4"
server_key_version: "encryptionkey@0"
I20260807 05:08:36.797578 21314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:36.823997 21314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:36.827960 21314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:36.828215 21314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:36.829121 21314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:36.830389 21314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:36.830462 21314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:36.830523 21314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:36.830559 21314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:36.841025 21314 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:34031
I20260807 05:08:36.841053 21446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:34031 every 8 connection(s)
I20260807 05:08:36.841984 21314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
Aug 07 05:08:36 dist-test-slave-1x4g krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079316, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.194@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:36.851639 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 21314
I20260807 05:08:36.851795 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:08:36.852106 1095 external_mini_cluster.cc:1428] Setting key 984e8b956cc9af3853541e0f0e1a460b
I20260807 05:08:36.855786 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:36.843753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:50265 (local address 127.1.17.254:40505)
0807 05:08:36.843920 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:36.843925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:36.844772 (+ 847us) server_negotiation.cc:407] Connection header received
0807 05:08:36.846088 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:36.846091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:36.846137 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:36.846211 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:36.847305 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.847811 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.848899 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.849113 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.849190 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:36.851626 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:36.851629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:36.851631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:36.851652 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:36.853313 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:36.853976 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:36.853981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:36.853982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:36.854085 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:36.854491 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:36.854496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:36.854500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:36.854714 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:36.854804 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:36.855439 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 05:08:36.855562 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:08:36.856657 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:36.844159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:40505 (local address 127.1.17.194:50265)
0807 05:08:36.844621 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:36.844664 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:36.845886 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:36.846372 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:36.846379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:36.846721 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:36.847176 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:36.847189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.847918 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.847920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:36.848797 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:36.848807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:36.849239 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:36.849241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:36.849379 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:36.849960 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:36.849976 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:36.851467 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:36.853456 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:36.853462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:36.853497 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:36.853850 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:36.854193 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:36.854199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:36.854204 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:36.854391 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:36.854825 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:36.854834 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:36.855152 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 05:08:36.855380 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 05:08:36.855732 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":128,"threads_started":1}
I20260807 05:08:36.857846 21447 heartbeater.cc:347] Connected to a master server at 127.1.17.254:40505
I20260807 05:08:36.858083 21447 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:36.858559 21447 heartbeater.cc:511] Master 127.1.17.254:40505 requested a full tablet report, sending...
I20260807 05:08:36.859601 21110 ts_manager.cc:194] Registered new tserver with Master: 8ab9e92919014daca78070546788398f (127.1.17.194:34031)
I20260807 05:08:36.860165 21110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.194@KRBTEST.COM'} at 127.1.17.194:50265
WARNING: no policy specified for kudu/127.1.17.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
Entry for principal kudu/127.1.17.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab.
I20260807 05:08:36.879412 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40505
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:36.988458 21452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:36.988672 21452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:36.988726 21452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:36.992764 21452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:36.992837 21452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:36.992939 21452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:08:36.997973 21452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.195.keytab
--principal=kudu/127.1.17.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:40505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:36.998900 21452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:36.999846 21452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:37.009047 21460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:37.009047 21459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:37.009125 21463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:37.009481 21452 server_base.cc:1034] running on GCE node
I20260807 05:08:37.009807 21452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:37.010473 21452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:37.011636 21452 hybrid_clock.cc:648] HybridClock initialized: now 1786079317011618 us; error 28 us; skew 500 ppm
Aug 07 05:08:37 dist-test-slave-1x4g krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079317, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:37.014338 21452 init.cc:377] Logged in from keytab as kudu/127.1.17.195@KRBTEST.COM (short username kudu)
I20260807 05:08:37.015259 21452 webserver.cc:460] Webserver started at http://127.1.17.195:38943/ using document root <none> and password file <none>
I20260807 05:08:37.015753 21452 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:37.015806 21452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:37.015961 21452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:37.017944 21452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "c512fa65b36f4d7a82926d78553db2ca"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "17db57e81b0e14d0ad2baa88b011f929"
server_key_iv: "d5ec25a94129609a83454bbd09de7285"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.018438 21452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "c512fa65b36f4d7a82926d78553db2ca"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "17db57e81b0e14d0ad2baa88b011f929"
server_key_iv: "d5ec25a94129609a83454bbd09de7285"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.021935 21452 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:37.024210 21470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.025519 21452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:37.025632 21452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "c512fa65b36f4d7a82926d78553db2ca"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "17db57e81b0e14d0ad2baa88b011f929"
server_key_iv: "d5ec25a94129609a83454bbd09de7285"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.025729 21452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:37.060972 21452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:37.063812 21452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:37.063964 21452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:37.064496 21452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:37.065459 21452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:37.065505 21452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.065544 21452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:37.065560 21452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.074167 21452 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:39769
I20260807 05:08:37.074198 21589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:39769 every 8 connection(s)
I20260807 05:08:37.074996 21452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:08:37.075428 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 21452
I20260807 05:08:37.075595 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:08:37.075955 1095 external_mini_cluster.cc:1428] Setting key 3df17dc231243efa870180a29a3bd303
Aug 07 05:08:37 dist-test-slave-1x4g krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079317, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.195@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:37.088405 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.077128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:33637 (local address 127.1.17.254:40505)
0807 05:08:37.077264 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.077268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.078189 (+ 921us) server_negotiation.cc:407] Connection header received
0807 05:08:37.079446 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.079449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.079502 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.079580 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:37.080845 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.081439 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.082706 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.082934 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.083043 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.085403 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:37.085406 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:37.085408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:37.085431 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:37.086638 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:37.087175 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:37.087177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:37.087179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:37.087250 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:37.087539 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:37.087541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:37.087542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:37.087697 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:37.087768 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.088201 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.088294 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:08:37.089272 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.077548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:40505 (local address 127.1.17.195:33637)
0807 05:08:37.078056 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.078111 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.079249 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.079748 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.079754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.080089 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:37.080685 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.080699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.081569 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.081572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.082571 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.082580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.083082 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.083086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.083246 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.083744 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:08:37.083759 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:37.085256 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:37.086830 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:37.086835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:37.086846 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:37.087087 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:37.087327 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:08:37.087330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:08:37.087331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:08:37.087463 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:08:37.087780 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:37.087785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:37.087983 (+ 198us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.088124 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.088342 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":119,"threads_started":1}
I20260807 05:08:37.090359 21590 heartbeater.cc:347] Connected to a master server at 127.1.17.254:40505
I20260807 05:08:37.090588 21590 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:37.091032 21590 heartbeater.cc:511] Master 127.1.17.254:40505 requested a full tablet report, sending...
I20260807 05:08:37.092015 21110 ts_manager.cc:194] Registered new tserver with Master: c512fa65b36f4d7a82926d78553db2ca (127.1.17.195:39769)
I20260807 05:08:37.092561 21110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.17.195@KRBTEST.COM'} at 127.1.17.195:33637
I20260807 05:08:37.101676 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:08:37.110321 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.103611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.1.17.254:40505)
0807 05:08:37.103714 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.103717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.103912 (+ 195us) server_negotiation.cc:407] Connection header received
0807 05:08:37.103972 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.103974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.104010 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.104088 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:37.104753 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.105294 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.106106 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.106303 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.106392 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.107680 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:37.107683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:37.107684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:37.107703 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:37.108973 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:37.109326 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:37.109328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:37.109329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:37.109427 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:37.109656 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:37.109658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:37.109659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:37.109847 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:37.109949 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.110136 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.110219 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:08:37.112982 21110 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:08:37.115998 21110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:08:37.130087 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.124061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:34031 (local address 127.0.0.1:38896)
0807 05:08:37.124517 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.124587 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.124711 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.125594 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.125600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.125623 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:37.125744 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.125751 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.127130 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.127133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.128578 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.128589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.129596 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.129599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.129703 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.129737 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.129835 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.129953 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":24,"thread_start_us":83,"threads_started":1}
I20260807 05:08:37.130106 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.124249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38896 (local address 127.1.17.194:34031)
0807 05:08:37.125063 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.125076 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.125100 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:08:37.125175 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.125181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.125389 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.125600 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:37.125867 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.126990 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.128722 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.129473 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.129596 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.129742 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.129881 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.129981 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":135,"threads_started":1}
I20260807 05:08:37.130368 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.124480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39421 (local address 127.0.0.1:35650)
0807 05:08:37.124929 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.124945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.125029 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.126263 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.126266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.126285 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:37.126396 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.126401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.127815 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.127819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.128890 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.128900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.129893 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.129897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.130036 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.130105 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.130172 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.130238 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":90,"thread_start_us":130,"threads_started":1}
I20260807 05:08:37.130604 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.124954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.1.17.193:39421)
0807 05:08:37.125728 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.125744 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.125767 (+ 23us) server_negotiation.cc:407] Connection header received
0807 05:08:37.125866 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.125871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.126094 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.126540 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:37.126688 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.127688 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.129001 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.129747 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.130164 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.130297 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.130344 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.130452 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":69,"threads_started":1}
I20260807 05:08:37.130842 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.124836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:39769 (local address 127.0.0.1:50146)
0807 05:08:37.125354 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.125386 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.125472 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.126145 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.126147 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.126171 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:37.126280 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.126289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.127289 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.127292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.128585 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.128590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.129893 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.129897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.130033 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.130601 (+ 568us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.130661 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.130733 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":107,"thread_start_us":97,"threads_started":1}
I20260807 05:08:37.131762 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.125308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.1.17.195:39769)
0807 05:08:37.125713 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.125725 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.125747 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:08:37.125816 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.125823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.126015 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.126169 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:37.126415 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.127125 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.128930 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.129747 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.131207 (+ 1460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.131469 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.131522 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.131615 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":140,"threads_started":1}
I20260807 05:08:37.133723 21379 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee859d21d95489f9d0ca4c7d2613dbf (DEFAULT_TABLE table=test-table [id=be3e0d69b67a4c3ab3a8d37257444421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:37.133978 21241 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee859d21d95489f9d0ca4c7d2613dbf (DEFAULT_TABLE table=test-table [id=be3e0d69b67a4c3ab3a8d37257444421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:37.134049 21520 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee859d21d95489f9d0ca4c7d2613dbf (DEFAULT_TABLE table=test-table [id=be3e0d69b67a4c3ab3a8d37257444421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:37.134882 21241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee859d21d95489f9d0ca4c7d2613dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:37.134876 21379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee859d21d95489f9d0ca4c7d2613dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:37.135142 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee859d21d95489f9d0ca4c7d2613dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:37.141042 21608 tablet_bootstrap.cc:492] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Bootstrap starting.
I20260807 05:08:37.142488 21609 tablet_bootstrap.cc:492] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52: Bootstrap starting.
I20260807 05:08:37.143515 21608 tablet_bootstrap.cc:654] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:37.144224 21609 tablet_bootstrap.cc:654] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:37.144583 21608 log.cc:824] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:37.145022 21609 log.cc:824] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:37.145102 21610 tablet_bootstrap.cc:492] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Bootstrap starting.
I20260807 05:08:37.146732 21609 tablet_bootstrap.cc:492] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52: No bootstrap required, opened a new log
I20260807 05:08:37.146940 21609 ts_tablet_manager.cc:1397] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 05:08:37.147053 21608 tablet_bootstrap.cc:492] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: No bootstrap required, opened a new log
I20260807 05:08:37.147256 21608 ts_tablet_manager.cc:1397] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 05:08:37.147523 21610 tablet_bootstrap.cc:654] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:37.148279 21610 log.cc:824] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:37.150125 21610 tablet_bootstrap.cc:492] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: No bootstrap required, opened a new log
I20260807 05:08:37.150292 21610 ts_tablet_manager.cc:1397] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 05:08:37.150496 21609 raft_consensus.cc:348] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.150712 21609 raft_consensus.cc:374] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:37.150760 21609 raft_consensus.cc:729] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e12a17bc3254833bb987225bd201d52, State: Initialized, Role: FOLLOWER
I20260807 05:08:37.150897 21608 raft_consensus.cc:348] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.151160 21608 raft_consensus.cc:374] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:37.151216 21608 raft_consensus.cc:729] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c512fa65b36f4d7a82926d78553db2ca, State: Initialized, Role: FOLLOWER
I20260807 05:08:37.151237 21609 consensus_queue.cc:260] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.151665 21608 consensus_queue.cc:260] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.151872 21308 heartbeater.cc:503] Master 127.1.17.254:40505 was elected leader, sending a full tablet report...
I20260807 05:08:37.152488 21609 ts_tablet_manager.cc:1428] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260807 05:08:37.152606 21608 ts_tablet_manager.cc:1428] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:08:37.152999 21590 heartbeater.cc:503] Master 127.1.17.254:40505 was elected leader, sending a full tablet report...
I20260807 05:08:37.153937 21610 raft_consensus.cc:348] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.154217 21610 raft_consensus.cc:374] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:37.154274 21610 raft_consensus.cc:729] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ab9e92919014daca78070546788398f, State: Initialized, Role: FOLLOWER
I20260807 05:08:37.154717 21610 consensus_queue.cc:260] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.155445 21614 raft_consensus.cc:482] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:37.155598 21614 raft_consensus.cc:504] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.155658 21610 ts_tablet_manager.cc:1428] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 05:08:37.155966 21447 heartbeater.cc:503] Master 127.1.17.254:40505 was elected leader, sending a full tablet report...
I20260807 05:08:37.156700 21614 leader_election.cc:288] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ab9e92919014daca78070546788398f (127.1.17.194:34031), c512fa65b36f4d7a82926d78553db2ca (127.1.17.195:39769)
I20260807 05:08:37.162272 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.157850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:33909 (local address 127.1.17.195:39769)
0807 05:08:37.157981 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.157985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.158013 (+ 28us) server_negotiation.cc:407] Connection header received
0807 05:08:37.158056 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.158058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.158089 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.158168 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:37.158429 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.159149 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.160969 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.161632 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.161755 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.161804 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.162079 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.162147 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 05:08:37.162272 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.157135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:47337 (local address 127.1.17.194:34031)
0807 05:08:37.158101 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.158104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.158117 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:08:37.158154 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.158157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.158198 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.158271 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:37.158636 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.159319 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.160969 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.161632 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.161755 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.161804 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.162079 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.162147 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895}
I20260807 05:08:37.162328 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.157523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:39769 (local address 127.1.17.193:33909)
0807 05:08:37.157751 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.157765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.157853 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.158179 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.158182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.158197 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:37.158306 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.158313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.159245 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.159248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.160838 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.160849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.161759 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.161763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.161932 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.161975 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.162045 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.162118 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":156,"thread_start_us":95,"threads_started":1}
I20260807 05:08:37.162792 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.157137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:34031 (local address 127.1.17.193:47337)
0807 05:08:37.157441 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.157460 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.157571 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.158372 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.158375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.158391 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:37.158521 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.158529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.159424 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.159427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.160838 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.160849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.161759 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.161763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.161916 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.161965 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.162045 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.162170 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":147,"thread_start_us":97,"threads_started":1}
I20260807 05:08:37.162854 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee859d21d95489f9d0ca4c7d2613dbf" candidate_uuid: "2e12a17bc3254833bb987225bd201d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c512fa65b36f4d7a82926d78553db2ca" is_pre_election: true
I20260807 05:08:37.163136 21542 raft_consensus.cc:2454] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e12a17bc3254833bb987225bd201d52 in term 0.
I20260807 05:08:37.163651 21193 leader_election.cc:302] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e12a17bc3254833bb987225bd201d52, c512fa65b36f4d7a82926d78553db2ca; no voters:
I20260807 05:08:37.163885 21400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee859d21d95489f9d0ca4c7d2613dbf" candidate_uuid: "2e12a17bc3254833bb987225bd201d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ab9e92919014daca78070546788398f" is_pre_election: true
I20260807 05:08:37.163933 21614 raft_consensus.cc:2781] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:08:37.164009 21614 raft_consensus.cc:482] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:08:37.164072 21614 raft_consensus.cc:3037] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:37.164165 21400 raft_consensus.cc:2454] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e12a17bc3254833bb987225bd201d52 in term 0.
I20260807 05:08:37.165165 21614 raft_consensus.cc:504] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.165431 21614 leader_election.cc:288] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [CANDIDATE]: Term 1 election: Requested vote from peers 8ab9e92919014daca78070546788398f (127.1.17.194:34031), c512fa65b36f4d7a82926d78553db2ca (127.1.17.195:39769)
I20260807 05:08:37.165774 21400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee859d21d95489f9d0ca4c7d2613dbf" candidate_uuid: "2e12a17bc3254833bb987225bd201d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ab9e92919014daca78070546788398f"
I20260807 05:08:37.165796 21542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee859d21d95489f9d0ca4c7d2613dbf" candidate_uuid: "2e12a17bc3254833bb987225bd201d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c512fa65b36f4d7a82926d78553db2ca"
I20260807 05:08:37.165906 21400 raft_consensus.cc:3037] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:37.165894 21542 raft_consensus.cc:3037] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:37.166867 21400 raft_consensus.cc:2454] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e12a17bc3254833bb987225bd201d52 in term 1.
I20260807 05:08:37.166889 21542 raft_consensus.cc:2454] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e12a17bc3254833bb987225bd201d52 in term 1.
I20260807 05:08:37.167176 21193 leader_election.cc:302] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e12a17bc3254833bb987225bd201d52, c512fa65b36f4d7a82926d78553db2ca; no voters:
I20260807 05:08:37.167337 21614 raft_consensus.cc:2781] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:37.167450 21614 raft_consensus.cc:686] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 1 LEADER]: Becoming Leader. State: Replica: 2e12a17bc3254833bb987225bd201d52, State: Running, Role: LEADER
I20260807 05:08:37.167745 21614 consensus_queue.cc:237] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } }
I20260807 05:08:37.170732 21110 catalog_manager.cc:5495] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e12a17bc3254833bb987225bd201d52 (127.1.17.193). New cstate: current_term: 1 leader_uuid: "2e12a17bc3254833bb987225bd201d52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e12a17bc3254833bb987225bd201d52" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 39421 } health_report { overall_health: HEALTHY } } }
I20260807 05:08:37.180441 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.177371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.1.17.193:39421)
0807 05:08:37.177485 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.177488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.177520 (+ 32us) server_negotiation.cc:407] Connection header received
0807 05:08:37.177657 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.177660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.177706 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.177785 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:08:37.178009 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.178459 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.179204 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.179396 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.179466 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.179708 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:08:37.180070 (+ 362us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:08:37.180154 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.180243 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.180298 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 05:08:37.185856 21400 raft_consensus.cc:1261] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 1 FOLLOWER]: Refusing update from remote peer 2e12a17bc3254833bb987225bd201d52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:37.185856 21542 raft_consensus.cc:1261] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 1 FOLLOWER]: Refusing update from remote peer 2e12a17bc3254833bb987225bd201d52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:37.186569 21618 consensus_queue.cc:1035] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ab9e92919014daca78070546788398f" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:37.186733 21614 consensus_queue.cc:1035] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c512fa65b36f4d7a82926d78553db2ca" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:37.193687 21624 mvcc.cc:205] Tried to move back new op lower bound from 7315780883186438144 to 7315780883121778688. Current Snapshot: MvccSnapshot[applied={T|T < 7315780883186438144}]
I20260807 05:08:37.195426 21625 mvcc.cc:205] Tried to move back new op lower bound from 7315780883186438144 to 7315780883121778688. Current Snapshot: MvccSnapshot[applied={T|T < 7315780883186438144}]
I20260807 05:08:37.203086 21108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:37.203279 21108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:37.205682 21108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a577f16184de48a7bebbadfd4bcae3e0: Sending DeleteTablet for 3 replicas of tablet 6ee859d21d95489f9d0ca4c7d2613dbf
I20260807 05:08:37.206532 21241 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee859d21d95489f9d0ca4c7d2613dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:37 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:35650
I20260807 05:08:37.206552 21520 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee859d21d95489f9d0ca4c7d2613dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:37 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:50146
I20260807 05:08:37.207041 21633 tablet_replica.cc:330] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52: stopping tablet replica
I20260807 05:08:37.207041 21379 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee859d21d95489f9d0ca4c7d2613dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:37 UTC) from {username='kudu', principal='kudu/127.1.17.254@KRBTEST.COM'} at 127.0.0.1:38896
I20260807 05:08:37.207352 21633 raft_consensus.cc:2229] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:08:37.207505 21634 tablet_replica.cc:330] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: stopping tablet replica
I20260807 05:08:37.207717 21633 raft_consensus.cc:2258] T 6ee859d21d95489f9d0ca4c7d2613dbf P 2e12a17bc3254833bb987225bd201d52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:37.207881 21634 raft_consensus.cc:2229] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:37.208115 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 21176
I20260807 05:08:37.209199 21632 tablet_replica.cc:330] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: stopping tablet replica
I20260807 05:08:37.209532 21632 raft_consensus.cc:2229] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:37.209585 21634 raft_consensus.cc:2258] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:37.209857 21632 raft_consensus.cc:2258] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:37.211210 21634 ts_tablet_manager.cc:1905] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:37.211521 21632 ts_tablet_manager.cc:1905] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:37.214102 21634 ts_tablet_manager.cc:1918] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:37.214249 21634 log.cc:1196] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/6ee859d21d95489f9d0ca4c7d2613dbf
I20260807 05:08:37.214603 21634 ts_tablet_manager.cc:1939] T 6ee859d21d95489f9d0ca4c7d2613dbf P 8ab9e92919014daca78070546788398f: Deleting consensus metadata
I20260807 05:08:37.214814 21632 ts_tablet_manager.cc:1918] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:37.214922 21632 log.cc:1196] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/6ee859d21d95489f9d0ca4c7d2613dbf
I20260807 05:08:37.215270 21632 ts_tablet_manager.cc:1939] T 6ee859d21d95489f9d0ca4c7d2613dbf P c512fa65b36f4d7a82926d78553db2ca: Deleting consensus metadata
I20260807 05:08:37.215878 21094 catalog_manager.cc:4841] TS 8ab9e92919014daca78070546788398f (127.1.17.194:34031): tablet 6ee859d21d95489f9d0ca4c7d2613dbf (table test-table [id=be3e0d69b67a4c3ab3a8d37257444421]) successfully deleted
I20260807 05:08:37.216313 21094 catalog_manager.cc:4841] TS c512fa65b36f4d7a82926d78553db2ca (127.1.17.195:39769): tablet 6ee859d21d95489f9d0ca4c7d2613dbf (table test-table [id=be3e0d69b67a4c3ab3a8d37257444421]) successfully deleted
W20260807 05:08:37.218776 21093 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:39421 (error 108)
I20260807 05:08:37.219064 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 21314
I20260807 05:08:37.221366 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.221006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:39421 (local address 127.0.0.1:35664)
0807 05:08:37.221179 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.221265 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:39421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 05:08:37.221550 21093 catalog_manager.cc:4584] TS 2e12a17bc3254833bb987225bd201d52 (127.1.17.193:39421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ee859d21d95489f9d0ca4c7d2613dbf: Network error: Client connection negotiation failed: client connection to 127.1.17.193:39421: connect: Connection refused (error 111)
I20260807 05:08:37.226310 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 21452
I20260807 05:08:37.233310 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 21075
2026-08-07T05:08:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2249 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T05:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:37Z Disabled control of system clock
I20260807 05:08:37.266361 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34011
--webserver_interface=127.1.17.254
--webserver_port=0
--builtin_ntp_servers=127.1.17.212:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:34011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:08:37.375903 21639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:37.376119 21639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:37.376150 21639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:37.379915 21639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:37.379961 21639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:37.379976 21639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:37.379992 21639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:37.380007 21639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:37.384531 21639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:34011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:34011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:37.385478 21639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:37.387055 21639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:37.395474 21644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:37.395442 21647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:37.395464 21645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:37.395612 21639 server_base.cc:1034] running on GCE node
I20260807 05:08:37.396167 21639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:37.397279 21639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:37.398468 21639 hybrid_clock.cc:648] HybridClock initialized: now 1786079317398449 us; error 29 us; skew 500 ppm
I20260807 05:08:37.400206 21639 webserver.cc:460] Webserver started at http://127.1.17.254:38657/ using document root <none> and password file <none>
I20260807 05:08:37.400760 21639 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:37.400934 21639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:37.401189 21639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:37.403573 21639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "b40a8dc4d89d47478cdfaa71d0446dd3"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "572a5e10d4cad96e4eaf7ce2aea3bf5f"
server_key_iv: "bc7625fcaf71c3d7e5ccbfbf2fdabd81"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.404129 21639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "b40a8dc4d89d47478cdfaa71d0446dd3"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "572a5e10d4cad96e4eaf7ce2aea3bf5f"
server_key_iv: "bc7625fcaf71c3d7e5ccbfbf2fdabd81"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.407698 21639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:08:37.410100 21653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.411288 21639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:08:37.411440 21639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "b40a8dc4d89d47478cdfaa71d0446dd3"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "572a5e10d4cad96e4eaf7ce2aea3bf5f"
server_key_iv: "bc7625fcaf71c3d7e5ccbfbf2fdabd81"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.411551 21639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:37.432192 21639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:37.433007 21639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:37.433223 21639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:37.440563 21639 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:34011
I20260807 05:08:37.440596 21707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:34011 every 8 connection(s)
I20260807 05:08:37.441610 21639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:37.444800 21708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:37.450008 21708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3: Bootstrap starting.
I20260807 05:08:37.451743 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 21639
I20260807 05:08:37.451895 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:37.452178 1095 external_mini_cluster.cc:1428] Setting key 7d00743afee0f344648556c884899575
I20260807 05:08:37.452433 21708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:37.453264 21708 log.cc:824] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:37.455143 21708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3: No bootstrap required, opened a new log
I20260807 05:08:37.459336 21708 raft_consensus.cc:348] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34011 } }
I20260807 05:08:37.459549 21708 raft_consensus.cc:374] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:37.459607 21708 raft_consensus.cc:729] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40a8dc4d89d47478cdfaa71d0446dd3, State: Initialized, Role: FOLLOWER
I20260807 05:08:37.460104 21708 consensus_queue.cc:260] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34011 } }
I20260807 05:08:37.460275 21708 raft_consensus.cc:388] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:37.460340 21708 raft_consensus.cc:482] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:37.460459 21708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:37.461234 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.453305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.1.17.254:34011)
0807 05:08:37.453958 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.453976 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.454010 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:08:37.454580 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.454596 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.454908 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.455226 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:37.455580 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.456559 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.457887 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.458361 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.458476 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.459327 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:37.459336 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:37.459348 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:37.459803 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:37.459861 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:37.459887 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:37.459963 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.460121 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.460399 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":158,"threads_started":1}
I20260807 05:08:37.461423 21708 raft_consensus.cc:504] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34011 } }
I20260807 05:08:37.461769 21708 leader_election.cc:302] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b40a8dc4d89d47478cdfaa71d0446dd3; no voters:
I20260807 05:08:37.462075 21708 leader_election.cc:288] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:37.462182 21714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:37.462389 21714 raft_consensus.cc:686] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [term 1 LEADER]: Becoming Leader. State: Replica: b40a8dc4d89d47478cdfaa71d0446dd3, State: Running, Role: LEADER
I20260807 05:08:37.462677 21714 consensus_queue.cc:237] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34011 } }
I20260807 05:08:37.463231 21708 sys_catalog.cc:564] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:37.463788 21716 sys_catalog.cc:455] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b40a8dc4d89d47478cdfaa71d0446dd3. Latest consensus state: current_term: 1 leader_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34011 } } }
I20260807 05:08:37.463909 21716 sys_catalog.cc:458] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:37.464339 21719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:37.464291 21715 sys_catalog.cc:455] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b40a8dc4d89d47478cdfaa71d0446dd3" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 34011 } } }
I20260807 05:08:37.465298 21715 sys_catalog.cc:458] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:37.468139 21719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:37.473742 21719 catalog_manager.cc:1347] Generated new cluster ID: 5fb0aea3bc97465ba01d055697e82b33
I20260807 05:08:37.473827 21719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:37.484047 21719 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:37.484910 21719 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:37.489419 21719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3: Generated new TSK 0
I20260807 05:08:37.490089 21719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:37.496241 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.193:0
--local_ip_for_outbound_sockets=127.1.17.193
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34011
--builtin_ntp_servers=127.1.17.212:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 05:08:37.609496 21733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:37.609766 21733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:37.609829 21733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:37.613828 21733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:37.613905 21733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:37.613999 21733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.193
I20260807 05:08:37.618695 21733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.17.193
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.193
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:37.619666 21733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:37.621029 21733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:37.630497 21740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:37.630420 21741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:37.630560 21733 server_base.cc:1034] running on GCE node
W20260807 05:08:37.630450 21743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:37.631063 21733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:37.631685 21733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:37.632846 21733 hybrid_clock.cc:648] HybridClock initialized: now 1786079317632827 us; error 28 us; skew 500 ppm
I20260807 05:08:37.634693 21733 webserver.cc:460] Webserver started at http://127.1.17.193:46663/ using document root <none> and password file <none>
I20260807 05:08:37.635270 21733 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:37.635358 21733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:37.635581 21733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:37.637661 21733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data/instance:
uuid: "f309b888571a4e0289a5db6c972fa364"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "f10776bc802bc1e6f2b8cf65bba1a126"
server_key_iv: "129b1bb2d3a65a31764cd9950519611b"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.638262 21733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance:
uuid: "f309b888571a4e0289a5db6c972fa364"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "f10776bc802bc1e6f2b8cf65bba1a126"
server_key_iv: "129b1bb2d3a65a31764cd9950519611b"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.641772 21733 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:37.644201 21749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.645416 21733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 05:08:37.645539 21733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/wal
uuid: "f309b888571a4e0289a5db6c972fa364"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "f10776bc802bc1e6f2b8cf65bba1a126"
server_key_iv: "129b1bb2d3a65a31764cd9950519611b"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.645642 21733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:37.673143 21733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:37.673875 21733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:37.674082 21733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:37.674675 21733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:37.675638 21733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:37.675717 21733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.675805 21733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:37.675853 21733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.684705 21733 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.193:46137
I20260807 05:08:37.684725 21868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.193:46137 every 8 connection(s)
I20260807 05:08:37.685688 21733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/data/info.pb
I20260807 05:08:37.693696 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 21733
I20260807 05:08:37.693837 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-0/wal/instance
I20260807 05:08:37.694139 1095 external_mini_cluster.cc:1428] Setting key db2d5c96aa01ebccd892e54f918b8b0c
I20260807 05:08:37.694373 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.687373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:53559 (local address 127.1.17.254:34011)
0807 05:08:37.687497 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.687501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.688470 (+ 969us) server_negotiation.cc:407] Connection header received
0807 05:08:37.689778 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.689781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.689826 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.689897 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:37.690669 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.691223 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.692289 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.692527 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.692600 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.693836 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:37.693839 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:37.693841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:37.693852 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:37.693888 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:37.693901 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:37.694001 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.694169 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.694232 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 05:08:37.695407 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.687698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34011 (local address 127.1.17.193:53559)
0807 05:08:37.688301 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.688339 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.689584 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.690025 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.690031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.690104 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:37.690542 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.690552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.691334 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.691336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.692183 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.692191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.692646 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.692649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.692822 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.693500 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:37.693503 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:37.693570 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:37.693650 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:37.694009 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:37.694020 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:37.694027 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.694162 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.694417 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":113,"threads_started":1}
I20260807 05:08:37.696276 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.194:0
--local_ip_for_outbound_sockets=127.1.17.194
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34011
--builtin_ntp_servers=127.1.17.212:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:08:37.696578 21869 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34011
I20260807 05:08:37.696866 21869 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:37.697367 21869 heartbeater.cc:511] Master 127.1.17.254:34011 requested a full tablet report, sending...
I20260807 05:08:37.698866 21670 ts_manager.cc:194] Registered new tserver with Master: f309b888571a4e0289a5db6c972fa364 (127.1.17.193:46137)
I20260807 05:08:37.700021 21670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.193:53559
W20260807 05:08:37.806635 21872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:37.806846 21872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:37.806910 21872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:37.810828 21872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:37.810873 21872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:37.810959 21872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.194
I20260807 05:08:37.815665 21872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.17.194
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.194
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:37.816624 21872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:37.817943 21872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:37.826967 21878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:37.827045 21877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:37.826982 21880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:37.827322 21872 server_base.cc:1034] running on GCE node
I20260807 05:08:37.827755 21872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:37.828313 21872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:37.829496 21872 hybrid_clock.cc:648] HybridClock initialized: now 1786079317829484 us; error 23 us; skew 500 ppm
I20260807 05:08:37.831146 21872 webserver.cc:460] Webserver started at http://127.1.17.194:36251/ using document root <none> and password file <none>
I20260807 05:08:37.831714 21872 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:37.831805 21872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:37.832031 21872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:37.834012 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data/instance:
uuid: "8569bceca42b46b09da381c77a1b2673"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "21b9344fcf1197fd9bc0ce988abdc0a0"
server_key_iv: "9ce91429645ede8a3593b4e26e800d86"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.834570 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance:
uuid: "8569bceca42b46b09da381c77a1b2673"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "21b9344fcf1197fd9bc0ce988abdc0a0"
server_key_iv: "9ce91429645ede8a3593b4e26e800d86"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.838171 21872 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:37.840562 21887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.841722 21872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:37.841852 21872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal
uuid: "8569bceca42b46b09da381c77a1b2673"
format_stamp: "Formatted at 2026-08-07 05:08:37 on dist-test-slave-1x4g"
server_key: "21b9344fcf1197fd9bc0ce988abdc0a0"
server_key_iv: "9ce91429645ede8a3593b4e26e800d86"
server_key_version: "encryptionkey@0"
I20260807 05:08:37.841959 21872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:37.855387 21872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:37.855974 21872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:37.856150 21872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:37.856700 21872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:37.857573 21872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:37.857647 21872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.857717 21872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:37.857760 21872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:37.866660 21872 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.194:36437
I20260807 05:08:37.866704 22006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.194:36437 every 8 connection(s)
I20260807 05:08:37.867463 21872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/data/info.pb
I20260807 05:08:37.873409 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 21872
I20260807 05:08:37.873593 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal/instance
I20260807 05:08:37.873894 1095 external_mini_cluster.cc:1428] Setting key 0b931e65e53bbdd7b1eae4b2a097ea8a
I20260807 05:08:37.876024 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.17.195:0
--local_ip_for_outbound_sockets=127.1.17.195
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34011
--builtin_ntp_servers=127.1.17.212:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 05:08:37.876767 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.869108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.194:45221 (local address 127.1.17.254:34011)
0807 05:08:37.869265 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:37.869270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:37.870273 (+ 1003us) server_negotiation.cc:407] Connection header received
0807 05:08:37.871581 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:37.871583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:37.871627 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:37.871699 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:37.872515 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.873128 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.874688 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.875012 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.875114 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.876256 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:37.876261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:37.876266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:37.876282 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:37.876308 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:37.876320 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:37.876364 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:37.876578 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 05:08:37.876654 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":64}
I20260807 05:08:37.877941 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:37.869477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34011 (local address 127.1.17.194:45221)
0807 05:08:37.870120 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:37.870158 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:37.871395 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:37.871845 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:37.871851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:37.871932 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:37.872393 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.872405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.873259 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.873262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.874550 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:37.874562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:37.875183 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:37.875188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:37.875380 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:37.875959 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:37.875961 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:37.876009 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:37.876084 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:37.876429 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:37.876440 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:37.876448 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 05:08:37.876558 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 05:08:37.876936 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
I20260807 05:08:37.879110 22007 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34011
I20260807 05:08:37.879341 22007 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:37.879815 22007 heartbeater.cc:511] Master 127.1.17.254:34011 requested a full tablet report, sending...
I20260807 05:08:37.880862 21670 ts_manager.cc:194] Registered new tserver with Master: 8569bceca42b46b09da381c77a1b2673 (127.1.17.194:36437)
I20260807 05:08:37.881740 21670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.194:45221
W20260807 05:08:37.987771 22010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:37.988019 22010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:37.988054 22010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:37.992884 22010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:37.992929 22010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:37.993036 22010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.17.195
I20260807 05:08:37.997706 22010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.17.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.17.195
--webserver_port=0
--tserver_master_addrs=127.1.17.254:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.17.195
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:37.998734 22010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:38.000094 22010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:38.009570 22015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:38.009569 22016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:38.009553 22018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:38.009862 22010 server_base.cc:1034] running on GCE node
I20260807 05:08:38.010325 22010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:38.010982 22010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:38.012174 22010 hybrid_clock.cc:648] HybridClock initialized: now 1786079318012162 us; error 34 us; skew 500 ppm
I20260807 05:08:38.014001 22010 webserver.cc:460] Webserver started at http://127.1.17.195:45503/ using document root <none> and password file <none>
I20260807 05:08:38.014688 22010 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:38.014776 22010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:38.015012 22010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:38.017174 22010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data/instance:
uuid: "1f516fb557354b0fbe0608dbbfa01819"
format_stamp: "Formatted at 2026-08-07 05:08:38 on dist-test-slave-1x4g"
server_key: "2473a98184f4b98fcb710b428aac75ae"
server_key_iv: "a1f8cd67ebf7a50a676453f62f0e9415"
server_key_version: "encryptionkey@0"
I20260807 05:08:38.017729 22010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance:
uuid: "1f516fb557354b0fbe0608dbbfa01819"
format_stamp: "Formatted at 2026-08-07 05:08:38 on dist-test-slave-1x4g"
server_key: "2473a98184f4b98fcb710b428aac75ae"
server_key_iv: "a1f8cd67ebf7a50a676453f62f0e9415"
server_key_version: "encryptionkey@0"
I20260807 05:08:38.021417 22010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:38.024046 22024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:38.025357 22010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:38.025501 22010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal
uuid: "1f516fb557354b0fbe0608dbbfa01819"
format_stamp: "Formatted at 2026-08-07 05:08:38 on dist-test-slave-1x4g"
server_key: "2473a98184f4b98fcb710b428aac75ae"
server_key_iv: "a1f8cd67ebf7a50a676453f62f0e9415"
server_key_version: "encryptionkey@0"
I20260807 05:08:38.025619 22010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:38.036428 22010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:38.037111 22010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:38.037305 22010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:38.037878 22010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 05:08:38.038766 22010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 05:08:38.038848 22010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:38.038928 22010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 05:08:38.038981 22010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:38.048046 22010 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.195:37649
I20260807 05:08:38.048071 22143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.195:37649 every 8 connection(s)
I20260807 05:08:38.049091 22010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/data/info.pb
I20260807 05:08:38.053498 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22010
I20260807 05:08:38.053612 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal/instance
I20260807 05:08:38.053936 1095 external_mini_cluster.cc:1428] Setting key 0e5983abaede93a5e15b2168a0865f84
I20260807 05:08:38.058928 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.051011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.195:43037 (local address 127.1.17.254:34011)
0807 05:08:38.051182 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.051187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.052249 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 05:08:38.053869 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.053872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.053920 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.054015 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:38.055000 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.055762 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.057153 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.057429 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.057518 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.058528 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:38.058531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:38.058533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:38.058545 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:38.058564 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:38.058572 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:38.058629 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.058769 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.058818 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 05:08:38.059774 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.051438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:34011 (local address 127.1.17.195:43037)
0807 05:08:38.052103 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:38.052144 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:38.053634 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:38.054260 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:38.054272 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:38.054377 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:08:38.054858 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.054872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.055892 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.055895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.057044 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.057054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.057565 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.057568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.057702 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.058278 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 05:08:38.058280 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:08:38.058319 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 05:08:38.058382 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:08:38.058644 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:08:38.058652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:08:38.058656 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 05:08:38.058749 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 05:08:38.058988 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
I20260807 05:08:38.060834 22144 heartbeater.cc:347] Connected to a master server at 127.1.17.254:34011
I20260807 05:08:38.061060 22144 heartbeater.cc:464] Registering TS with master...
I20260807 05:08:38.061597 22144 heartbeater.cc:511] Master 127.1.17.254:34011 requested a full tablet report, sending...
I20260807 05:08:38.062604 21670 ts_manager.cc:194] Registered new tserver with Master: 1f516fb557354b0fbe0608dbbfa01819 (127.1.17.195:37649)
I20260807 05:08:38.063139 21670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.17.195:43037
I20260807 05:08:38.067792 1095 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 05:08:38.073333 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.069794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.1.17.254:34011)
0807 05:08:38.069929 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.069932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.070136 (+ 204us) server_negotiation.cc:407] Connection header received
0807 05:08:38.070235 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.070237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.070274 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.070339 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:38.070609 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.071113 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.071930 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.072141 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.072236 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.073004 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:38.073008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:38.073010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:38.073021 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:38.073041 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:38.073049 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:38.073109 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.073183 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.073231 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:08:38.075855 21670 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 05:08:38.078931 21670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 05:08:38.093675 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.087267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46137 (local address 127.0.0.1:59636)
0807 05:08:38.087999 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:38.088010 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:38.088072 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:38.089261 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:38.089264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:38.089282 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:38.089382 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.089389 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.090446 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.090451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.091789 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.091800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.092740 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.092744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.093044 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.093145 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 05:08:38.093305 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 05:08:38.093516 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":119,"thread_start_us":107,"threads_started":1}
I20260807 05:08:38.093762 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.087993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59636 (local address 127.1.17.193:46137)
0807 05:08:38.088830 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.088842 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.088864 (+ 22us) server_negotiation.cc:407] Connection header received
0807 05:08:38.088941 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.088946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.089127 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.089288 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:38.089499 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.090335 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.091900 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.092607 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.092730 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.092949 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.093480 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.093587 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":102,"threads_started":1}
I20260807 05:08:38.093930 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.087207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:36437 (local address 127.0.0.1:45380)
0807 05:08:38.087856 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:38.087943 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:38.088072 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:38.089056 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:38.089063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:38.089081 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:38.089231 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.089237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.090816 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.090819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.092100 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.092108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.093288 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.093291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.093546 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.093572 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 05:08:38.093645 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 05:08:38.093711 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":217,"thread_start_us":78,"threads_started":1}
I20260807 05:08:38.094187 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.087546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:37649 (local address 127.0.0.1:42910)
0807 05:08:38.088963 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:38.088978 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:38.089066 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:38.089556 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:38.089559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:38.089576 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:38.089650 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.089655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.090953 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.090957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.092205 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.092216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.093272 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.093279 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.093894 (+ 615us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.093920 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 05:08:38.093998 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 05:08:38.094064 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1334,"mutex_wait_us":478,"thread_start_us":97,"threads_started":1}
I20260807 05:08:38.094473 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.087334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.1.17.194:36437)
0807 05:08:38.088438 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.088446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.088467 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:08:38.088567 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.088572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.088831 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.089372 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:38.089523 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.090697 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.092218 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.093153 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.093941 (+ 788us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.094145 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.094197 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.094302 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":160,"threads_started":1}
I20260807 05:08:38.095008 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.087993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.1.17.195:37649)
0807 05:08:38.088586 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.088600 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.089159 (+ 559us) server_negotiation.cc:407] Connection header received
0807 05:08:38.089237 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.089243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.089434 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.089586 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:38.089757 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.090846 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.092340 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.093122 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.093258 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.093584 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.094695 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.094797 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":106,"threads_started":1}
I20260807 05:08:38.097179 21939 tablet_service.cc:1467] Processing CreateTablet for tablet b3c5f415b89042c5bb7a468807d590ae (DEFAULT_TABLE table=test-table [id=9fb5caa474334609b20fbd829c175dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:38.097208 21798 tablet_service.cc:1467] Processing CreateTablet for tablet b3c5f415b89042c5bb7a468807d590ae (DEFAULT_TABLE table=test-table [id=9fb5caa474334609b20fbd829c175dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:38.097548 22075 tablet_service.cc:1467] Processing CreateTablet for tablet b3c5f415b89042c5bb7a468807d590ae (DEFAULT_TABLE table=test-table [id=9fb5caa474334609b20fbd829c175dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 05:08:38.098299 21939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c5f415b89042c5bb7a468807d590ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:38.098304 21798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c5f415b89042c5bb7a468807d590ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:38.098663 22075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c5f415b89042c5bb7a468807d590ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:38.105159 22161 tablet_bootstrap.cc:492] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Bootstrap starting.
I20260807 05:08:38.105861 22162 tablet_bootstrap.cc:492] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: Bootstrap starting.
I20260807 05:08:38.106343 22163 tablet_bootstrap.cc:492] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Bootstrap starting.
I20260807 05:08:38.107249 22161 tablet_bootstrap.cc:654] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:38.107978 22162 tablet_bootstrap.cc:654] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:38.107975 22161 log.cc:824] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:38.108273 22163 tablet_bootstrap.cc:654] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:38.108883 22162 log.cc:824] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:38.109051 22163 log.cc:824] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:38.110083 22161 tablet_bootstrap.cc:492] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: No bootstrap required, opened a new log
I20260807 05:08:38.110275 22161 ts_tablet_manager.cc:1397] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:08:38.111061 22163 tablet_bootstrap.cc:492] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: No bootstrap required, opened a new log
I20260807 05:08:38.111243 22163 ts_tablet_manager.cc:1397] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 05:08:38.113711 22162 tablet_bootstrap.cc:492] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: No bootstrap required, opened a new log
I20260807 05:08:38.113932 22162 ts_tablet_manager.cc:1397] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 05:08:38.114586 22161 raft_consensus.cc:348] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.114871 22161 raft_consensus.cc:374] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:38.114938 22161 raft_consensus.cc:729] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8569bceca42b46b09da381c77a1b2673, State: Initialized, Role: FOLLOWER
I20260807 05:08:38.115407 22161 consensus_queue.cc:260] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.115449 22163 raft_consensus.cc:348] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.115661 22163 raft_consensus.cc:374] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:38.115707 22163 raft_consensus.cc:729] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f516fb557354b0fbe0608dbbfa01819, State: Initialized, Role: FOLLOWER
I20260807 05:08:38.116160 22163 consensus_queue.cc:260] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.116439 22161 ts_tablet_manager.cc:1428] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:08:38.116541 22007 heartbeater.cc:503] Master 127.1.17.254:34011 was elected leader, sending a full tablet report...
I20260807 05:08:38.117105 22163 ts_tablet_manager.cc:1428] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 05:08:38.117542 22144 heartbeater.cc:503] Master 127.1.17.254:34011 was elected leader, sending a full tablet report...
W20260807 05:08:38.118301 22008 tablet.cc:2369] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:08:38.118111 22162 raft_consensus.cc:348] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.118647 22162 raft_consensus.cc:374] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:38.118696 22162 raft_consensus.cc:729] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f309b888571a4e0289a5db6c972fa364, State: Initialized, Role: FOLLOWER
I20260807 05:08:38.119098 22162 consensus_queue.cc:260] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.119760 21869 heartbeater.cc:503] Master 127.1.17.254:34011 was elected leader, sending a full tablet report...
I20260807 05:08:38.119979 22162 ts_tablet_manager.cc:1428] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 05:08:38.186748 21870 tablet.cc:2369] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 05:08:38.237742 22169 raft_consensus.cc:482] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 05:08:38.237949 22169 raft_consensus.cc:504] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.239055 22169 leader_election.cc:288] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8569bceca42b46b09da381c77a1b2673 (127.1.17.194:36437), 1f516fb557354b0fbe0608dbbfa01819 (127.1.17.195:37649)
I20260807 05:08:38.244707 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.239439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:44979 (local address 127.1.17.195:37649)
0807 05:08:38.239592 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.239597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.239863 (+ 266us) server_negotiation.cc:407] Connection header received
0807 05:08:38.240031 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.240034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.240104 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.240199 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:38.240569 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.241417 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.243344 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.243983 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.244095 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.244145 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.244495 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.244572 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:08:38.244725 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.239579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.17.193:55495 (local address 127.1.17.194:36437)
0807 05:08:38.239703 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.239708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.239981 (+ 273us) server_negotiation.cc:407] Connection header received
0807 05:08:38.240104 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.240108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.240165 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.240241 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 05:08:38.240569 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.241418 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.243344 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.243967 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.244095 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.244145 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.244495 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.244572 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:08:38.244796 22170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.239339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.195:37649 (local address 127.1.17.193:44979)
0807 05:08:38.239764 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:38.239779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:38.239903 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:38.240245 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:38.240249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:38.240270 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:38.240456 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.240462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.241551 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.241555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.243204 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.243217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.244101 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.244105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.244333 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.244376 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 05:08:38.244469 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 05:08:38.244582 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":261,"thread_start_us":102,"threads_started":1}
I20260807 05:08:38.244802 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.239339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.194:36437 (local address 127.1.17.193:55495)
0807 05:08:38.239856 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:38.239869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:08:38.239992 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:08:38.240265 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:08:38.240268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:08:38.240278 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 05:08:38.240449 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.240457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.241551 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.241555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.243204 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:08:38.243217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.244101 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.244105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:08:38.244323 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.244376 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 05:08:38.244462 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 05:08:38.244582 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":482,"thread_start_us":106,"threads_started":1}
I20260807 05:08:38.245663 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c5f415b89042c5bb7a468807d590ae" candidate_uuid: "f309b888571a4e0289a5db6c972fa364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569bceca42b46b09da381c77a1b2673" is_pre_election: true
I20260807 05:08:38.245692 22095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c5f415b89042c5bb7a468807d590ae" candidate_uuid: "f309b888571a4e0289a5db6c972fa364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f516fb557354b0fbe0608dbbfa01819" is_pre_election: true
I20260807 05:08:38.245978 22095 raft_consensus.cc:2454] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f309b888571a4e0289a5db6c972fa364 in term 0.
I20260807 05:08:38.246007 21959 raft_consensus.cc:2454] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f309b888571a4e0289a5db6c972fa364 in term 0.
I20260807 05:08:38.246486 21753 leader_election.cc:302] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f516fb557354b0fbe0608dbbfa01819, f309b888571a4e0289a5db6c972fa364; no voters:
I20260807 05:08:38.246769 22169 raft_consensus.cc:2781] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 05:08:38.246850 22169 raft_consensus.cc:482] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 05:08:38.246881 22169 raft_consensus.cc:3037] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:38.248026 22169 raft_consensus.cc:504] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.248387 22169 leader_election.cc:288] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [CANDIDATE]: Term 1 election: Requested vote from peers 8569bceca42b46b09da381c77a1b2673 (127.1.17.194:36437), 1f516fb557354b0fbe0608dbbfa01819 (127.1.17.195:37649)
I20260807 05:08:38.248742 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c5f415b89042c5bb7a468807d590ae" candidate_uuid: "f309b888571a4e0289a5db6c972fa364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8569bceca42b46b09da381c77a1b2673"
I20260807 05:08:38.248868 22095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c5f415b89042c5bb7a468807d590ae" candidate_uuid: "f309b888571a4e0289a5db6c972fa364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f516fb557354b0fbe0608dbbfa01819"
I20260807 05:08:38.248903 21959 raft_consensus.cc:3037] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:38.249027 22095 raft_consensus.cc:3037] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:38.250072 22095 raft_consensus.cc:2454] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f309b888571a4e0289a5db6c972fa364 in term 1.
I20260807 05:08:38.250087 21959 raft_consensus.cc:2454] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f309b888571a4e0289a5db6c972fa364 in term 1.
I20260807 05:08:38.250419 21752 leader_election.cc:302] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f516fb557354b0fbe0608dbbfa01819, 8569bceca42b46b09da381c77a1b2673, f309b888571a4e0289a5db6c972fa364; no voters:
I20260807 05:08:38.250658 22169 raft_consensus.cc:2781] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:38.250869 22169 raft_consensus.cc:686] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 1 LEADER]: Becoming Leader. State: Replica: f309b888571a4e0289a5db6c972fa364, State: Running, Role: LEADER
I20260807 05:08:38.251231 22169 consensus_queue.cc:237] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } }
I20260807 05:08:38.254411 21670 catalog_manager.cc:5495] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 reported cstate change: term changed from 0 to 1, leader changed from <none> to f309b888571a4e0289a5db6c972fa364 (127.1.17.193). New cstate: current_term: 1 leader_uuid: "f309b888571a4e0289a5db6c972fa364" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f309b888571a4e0289a5db6c972fa364" member_type: VOTER last_known_addr { host: "127.1.17.193" port: 46137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 } health_report { overall_health: UNKNOWN } } }
I20260807 05:08:38.280229 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.276557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59642 (local address 127.1.17.193:46137)
0807 05:08:38.276710 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:38.276715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:38.276729 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:08:38.276918 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:38.276920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:38.276987 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:38.277102 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:08:38.277353 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.277864 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.278750 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:38.278987 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:38.279100 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:38.279421 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:08:38.279840 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:08:38.279988 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:38.280026 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 05:08:38.280080 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:08:38.286345 21959 raft_consensus.cc:1261] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 1 FOLLOWER]: Refusing update from remote peer f309b888571a4e0289a5db6c972fa364: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:38.286343 22095 raft_consensus.cc:1261] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 1 FOLLOWER]: Refusing update from remote peer f309b888571a4e0289a5db6c972fa364: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:08:38.287086 22173 consensus_queue.cc:1035] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f516fb557354b0fbe0608dbbfa01819" member_type: VOTER last_known_addr { host: "127.1.17.195" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:38.287338 22169 consensus_queue.cc:1035] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8569bceca42b46b09da381c77a1b2673" member_type: VOTER last_known_addr { host: "127.1.17.194" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:08:38.297226 22177 mvcc.cc:205] Tried to move back new op lower bound from 7315780887693275136 to 7315780887559737344. Current Snapshot: MvccSnapshot[applied={T|T < 7315780887693275136 or (T in {7315780887693275136})}]
I20260807 05:08:38.304613 21668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:38.304762 21668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 05:08:38.307235 21668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b40a8dc4d89d47478cdfaa71d0446dd3: Sending DeleteTablet for 3 replicas of tablet b3c5f415b89042c5bb7a468807d590ae
I20260807 05:08:38.308019 21939 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c5f415b89042c5bb7a468807d590ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:38 UTC) from {username='slave'} at 127.0.0.1:45380
I20260807 05:08:38.308017 21798 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c5f415b89042c5bb7a468807d590ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:38 UTC) from {username='slave'} at 127.0.0.1:59636
I20260807 05:08:38.308198 22075 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c5f415b89042c5bb7a468807d590ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 05:08:38 UTC) from {username='slave'} at 127.0.0.1:42910
I20260807 05:08:38.308643 22188 tablet_replica.cc:330] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: stopping tablet replica
I20260807 05:08:38.308724 22187 tablet_replica.cc:330] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364: stopping tablet replica
I20260807 05:08:38.309068 22187 raft_consensus.cc:2229] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 1 LEADER]: Raft consensus shutting down.
I20260807 05:08:38.309069 22188 raft_consensus.cc:2229] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:38.309207 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 21733
I20260807 05:08:38.309460 22187 raft_consensus.cc:2258] T b3c5f415b89042c5bb7a468807d590ae P f309b888571a4e0289a5db6c972fa364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:38.309599 22188 raft_consensus.cc:2258] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:38.309777 22189 tablet_replica.cc:330] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: stopping tablet replica
I20260807 05:08:38.310276 22189 raft_consensus.cc:2229] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 05:08:38.310590 22189 raft_consensus.cc:2258] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 05:08:38.312734 22189 ts_tablet_manager.cc:1905] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:38.312929 22188 ts_tablet_manager.cc:1905] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 05:08:38.315359 22189 ts_tablet_manager.cc:1918] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:38.315480 22189 log.cc:1196] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-2/wal/wals/b3c5f415b89042c5bb7a468807d590ae
I20260807 05:08:38.315773 22189 ts_tablet_manager.cc:1939] T b3c5f415b89042c5bb7a468807d590ae P 1f516fb557354b0fbe0608dbbfa01819: Deleting consensus metadata
I20260807 05:08:38.316450 22188 ts_tablet_manager.cc:1918] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 05:08:38.316556 22188 log.cc:1196] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Deleting WAL directory at /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786079068077062-1095-0/minicluster-data/ts-1/wal/wals/b3c5f415b89042c5bb7a468807d590ae
I20260807 05:08:38.316943 22188 ts_tablet_manager.cc:1939] T b3c5f415b89042c5bb7a468807d590ae P 8569bceca42b46b09da381c77a1b2673: Deleting consensus metadata
I20260807 05:08:38.317037 21657 catalog_manager.cc:4841] TS 1f516fb557354b0fbe0608dbbfa01819 (127.1.17.195:37649): tablet b3c5f415b89042c5bb7a468807d590ae (table test-table [id=9fb5caa474334609b20fbd829c175dbb]) successfully deleted
I20260807 05:08:38.318125 21656 catalog_manager.cc:4841] TS 8569bceca42b46b09da381c77a1b2673 (127.1.17.194:36437): tablet b3c5f415b89042c5bb7a468807d590ae (table test-table [id=9fb5caa474334609b20fbd829c175dbb]) successfully deleted
W20260807 05:08:38.319662 21655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.17.193:46137 (error 108)
I20260807 05:08:38.319839 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 21872
I20260807 05:08:38.322660 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:38.320558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.193:46137 (local address 127.0.0.1:59658)
0807 05:08:38.322510 (+ 1952us) negotiation.cc:107] Waiting for socket to connect
0807 05:08:38.322586 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.193:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1845}
W20260807 05:08:38.322853 21655 catalog_manager.cc:4584] TS f309b888571a4e0289a5db6c972fa364 (127.1.17.193:46137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3c5f415b89042c5bb7a468807d590ae: Network error: Client connection negotiation failed: client connection to 127.1.17.193:46137: connect: Connection refused (error 111)
I20260807 05:08:38.327728 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22010
I20260807 05:08:38.334139 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 21639
2026-08-07T05:08:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1099 ms)
[----------] 6 tests from EncryptionPolicyTest (10729 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:38 dist-test-slave-1x4g krb5kdc[22193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:38 dist-test-slave-1x4g krb5kdc[22193](info): set up 2 sockets
Aug 07 05:08:38 dist-test-slave-1x4g krb5kdc[22193](info): commencing operation
krb5kdc: starting...
W20260807 05:08:39.411607 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:39 dist-test-slave-1x4g krb5kdc[22193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:39.538992 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34425
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:39.648566 22206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:39.648869 22206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:39.648924 22206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:39.652688 22206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:39.652732 22206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:39.652751 22206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:39.652766 22206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:39.652822 22206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:39.658030 22206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34425
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:39.658952 22206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:39.659881 22206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:39.668231 22214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:39.668320 22206 server_base.cc:1034] running on GCE node
W20260807 05:08:39.668334 22211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:39.668495 22212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:39.669144 22206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:39.670218 22206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:39.671392 22206 hybrid_clock.cc:648] HybridClock initialized: now 1786079319671362 us; error 39 us; skew 500 ppm
Aug 07 05:08:39 dist-test-slave-1x4g krb5kdc[22193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079319, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:39.679428 22206 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:39.680299 22206 webserver.cc:460] Webserver started at http://127.0.0.1:34629/ using document root <none> and password file <none>
I20260807 05:08:39.680943 22206 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:39.681041 22206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:39.681283 22206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:39.683368 22206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "fa1fb5ba9e6546359ac5c2b3046324dc"
format_stamp: "Formatted at 2026-08-07 05:08:39 on dist-test-slave-1x4g"
server_key: "2b09bd795234287ce24872515c1615ad"
server_key_iv: "87426379932b1154e87ec4ad4cb30ce3"
server_key_version: "encryptionkey@0"
I20260807 05:08:39.684026 22206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "fa1fb5ba9e6546359ac5c2b3046324dc"
format_stamp: "Formatted at 2026-08-07 05:08:39 on dist-test-slave-1x4g"
server_key: "2b09bd795234287ce24872515c1615ad"
server_key_iv: "87426379932b1154e87ec4ad4cb30ce3"
server_key_version: "encryptionkey@0"
I20260807 05:08:39.687794 22206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:08:39.690239 22221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:39.691431 22206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:08:39.691578 22206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "fa1fb5ba9e6546359ac5c2b3046324dc"
format_stamp: "Formatted at 2026-08-07 05:08:39 on dist-test-slave-1x4g"
server_key: "2b09bd795234287ce24872515c1615ad"
server_key_iv: "87426379932b1154e87ec4ad4cb30ce3"
server_key_version: "encryptionkey@0"
I20260807 05:08:39.691689 22206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:39.708524 22206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:39.711683 22206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:39.711925 22206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:39.719722 22206 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34425
I20260807 05:08:39.719731 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34425 every 8 connection(s)
I20260807 05:08:39.720669 22206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:39.723392 22278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:39.724640 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22206
I20260807 05:08:39.724778 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:39.725126 1095 external_mini_cluster.cc:1428] Setting key 01239753781e0256c862587b763c3f87
I20260807 05:08:39.729856 22278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc: Bootstrap starting.
I20260807 05:08:39.732352 22278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:39.733182 22278 log.cc:824] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc: Log is configured to *not* fsync() on all Append() calls
Aug 07 05:08:39 dist-test-slave-1x4g krb5kdc[22193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:08:39.735212 22278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc: No bootstrap required, opened a new log
I20260807 05:08:39.739889 22278 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34425 } }
I20260807 05:08:39.740171 22278 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:39.740206 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:39.726542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60023 (local address 127.0.0.1:34425)
0807 05:08:39.726958 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:39.726976 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:39.727018 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:08:39.727830 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:39.727853 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:39.728258 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:39.728612 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:39.729321 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:39.730718 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:39.731558 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:39.732228 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:39.732403 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:39.735167 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:39.735173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:39.735188 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:39.735216 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:39.737146 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:39.737655 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:39.737663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:39.737672 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:39.737819 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:39.738134 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:39.738138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:39.738139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:39.738630 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:39.738845 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:39.739039 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 05:08:39.739254 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":109,"threads_started":1}
I20260807 05:08:39.740242 22278 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa1fb5ba9e6546359ac5c2b3046324dc, State: Initialized, Role: FOLLOWER
I20260807 05:08:39.740679 22278 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34425 } }
I20260807 05:08:39.740862 22278 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:39.740958 22278 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:39.741078 22278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:39.742151 22278 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34425 } }
I20260807 05:08:39.742528 22278 leader_election.cc:302] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa1fb5ba9e6546359ac5c2b3046324dc; no voters:
I20260807 05:08:39.742856 22278 leader_election.cc:288] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:39.742991 22283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:39.743242 22283 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [term 1 LEADER]: Becoming Leader. State: Replica: fa1fb5ba9e6546359ac5c2b3046324dc, State: Running, Role: LEADER
I20260807 05:08:39.743583 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:39.743629 22283 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34425 } }
I20260807 05:08:39.744344 22278 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:39.744688 22283 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa1fb5ba9e6546359ac5c2b3046324dc. Latest consensus state: current_term: 1 leader_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34425 } } }
I20260807 05:08:39.744838 22283 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:39.745164 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1fb5ba9e6546359ac5c2b3046324dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34425 } } }
I20260807 05:08:39.745262 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc [sys.catalog]: This master's current role is: LEADER
W20260807 05:08:39.749780 22302 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:39.749863 22302 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:39.749931 22287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:39.750784 22287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:39.756135 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:39.746983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59365 (local address 127.0.0.1:34425)
0807 05:08:39.747083 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:39.747087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:39.747460 (+ 373us) server_negotiation.cc:407] Connection header received
0807 05:08:39.747566 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:39.747568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:39.747602 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:39.747665 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:39.748461 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:39.749236 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:39.750318 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:39.750644 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:39.750752 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:39.752112 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:39.752115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:39.752118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:39.752142 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:39.753857 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:39.754488 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:39.754491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:39.754493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:39.754579 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:39.755039 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:39.755043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:39.755044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:39.755252 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:39.755414 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:39.755819 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 05:08:39.755949 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:08:39.756325 22287 catalog_manager.cc:1347] Generated new cluster ID: 8118402ba779495893534fb9362adfb9
I20260807 05:08:39.756376 22287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:39.779318 22287 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:39.780491 22287 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:39.787488 22287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa1fb5ba9e6546359ac5c2b3046324dc: Generated new TSK 0
I20260807 05:08:39.788118 22287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:39.797900 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22206
W20260807 05:08:39.805598 22191 connection.cc:535] client connection to 127.0.0.1:34425 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34425): Connection reset by peer (error 104)
2026-08-07T05:08:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:39 dist-test-slave-1x4g krb5kdc[22310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:39 dist-test-slave-1x4g krb5kdc[22310](info): set up 2 sockets
Aug 07 05:08:39 dist-test-slave-1x4g krb5kdc[22310](info): commencing operation
krb5kdc: starting...
W20260807 05:08:40.853091 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:40 dist-test-slave-1x4g krb5kdc[22310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:40.984230 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:41.095099 22324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:41.095402 22324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:41.095487 22324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:41.099493 22324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:41.099582 22324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:41.099628 22324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:41.099689 22324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:41.099740 22324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:41.104861 22324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:41.105854 22324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:41.106873 22324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:41.115360 22330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:41.115363 22332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:41.115378 22329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:41.115746 22324 server_base.cc:1034] running on GCE node
I20260807 05:08:41.116307 22324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:41.117388 22324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:41.118554 22324 hybrid_clock.cc:648] HybridClock initialized: now 1786079321118529 us; error 40 us; skew 500 ppm
Aug 07 05:08:41 dist-test-slave-1x4g krb5kdc[22310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:41.121326 22324 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:41.122189 22324 webserver.cc:460] Webserver started at http://127.0.0.1:38541/ using document root <none> and password file <none>
I20260807 05:08:41.122745 22324 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:41.122800 22324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:41.123040 22324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:41.125003 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "dd49f7103b014f1f9a4332e4580566a2"
format_stamp: "Formatted at 2026-08-07 05:08:41 on dist-test-slave-1x4g"
server_key: "64da934e23d9f9e5cc97f9728378a1e7"
server_key_iv: "867e46442bdc567efd68b44c60d8b58e"
server_key_version: "encryptionkey@0"
I20260807 05:08:41.125576 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "dd49f7103b014f1f9a4332e4580566a2"
format_stamp: "Formatted at 2026-08-07 05:08:41 on dist-test-slave-1x4g"
server_key: "64da934e23d9f9e5cc97f9728378a1e7"
server_key_iv: "867e46442bdc567efd68b44c60d8b58e"
server_key_version: "encryptionkey@0"
I20260807 05:08:41.129083 22324 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:08:41.131510 22339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:41.132675 22324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:41.132835 22324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "dd49f7103b014f1f9a4332e4580566a2"
format_stamp: "Formatted at 2026-08-07 05:08:41 on dist-test-slave-1x4g"
server_key: "64da934e23d9f9e5cc97f9728378a1e7"
server_key_iv: "867e46442bdc567efd68b44c60d8b58e"
server_key_version: "encryptionkey@0"
I20260807 05:08:41.132948 22324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:41.151170 22324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:41.154103 22324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:41.154294 22324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:41.161679 22324 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33559
I20260807 05:08:41.161746 22392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33559 every 8 connection(s)
I20260807 05:08:41.162650 22324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:41.165357 22393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:41.170717 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2: Bootstrap starting.
I20260807 05:08:41.170922 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22324
I20260807 05:08:41.171074 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:41.171365 1095 external_mini_cluster.cc:1428] Setting key 4ef0b96409f3d3cfe6bdd358a9528bcd
I20260807 05:08:41.173269 22393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:41.173964 22393 log.cc:824] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:41.175913 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2: No bootstrap required, opened a new log
I20260807 05:08:41.180154 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:41.172617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45351 (local address 127.0.0.1:33559)
0807 05:08:41.173008 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:41.173031 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:41.173066 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:08:41.173712 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:41.173730 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:41.174046 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:41.174396 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:41.174832 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:41.175829 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:41.177083 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:41.177542 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:41.177657 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:41.178569 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:41.178574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:41.178585 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:41.178942 (+ 357us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:41.178991 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:41.179013 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:41.179108 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:41.179304 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 05:08:41.179456 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":151,"threads_started":1}
I20260807 05:08:41.180092 22393 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd49f7103b014f1f9a4332e4580566a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33559 } }
I20260807 05:08:41.180387 22393 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:41.180450 22393 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd49f7103b014f1f9a4332e4580566a2, State: Initialized, Role: FOLLOWER
I20260807 05:08:41.180904 22393 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd49f7103b014f1f9a4332e4580566a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33559 } }
I20260807 05:08:41.181053 22393 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:41.181116 22393 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:41.181210 22393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:41.182122 22393 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd49f7103b014f1f9a4332e4580566a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33559 } }
I20260807 05:08:41.182521 22393 leader_election.cc:302] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd49f7103b014f1f9a4332e4580566a2; no voters:
I20260807 05:08:41.182933 22393 leader_election.cc:288] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:41.183012 22399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:41.183130 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:41.183274 22399 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [term 1 LEADER]: Becoming Leader. State: Replica: dd49f7103b014f1f9a4332e4580566a2, State: Running, Role: LEADER
I20260807 05:08:41.183583 22399 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd49f7103b014f1f9a4332e4580566a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33559 } }
I20260807 05:08:41.184252 22393 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:41.184551 22399 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd49f7103b014f1f9a4332e4580566a2. Latest consensus state: current_term: 1 leader_uuid: "dd49f7103b014f1f9a4332e4580566a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd49f7103b014f1f9a4332e4580566a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33559 } } }
I20260807 05:08:41.184659 22399 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:41.185000 22400 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd49f7103b014f1f9a4332e4580566a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd49f7103b014f1f9a4332e4580566a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33559 } } }
I20260807 05:08:41.185137 22400 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2 [sys.catalog]: This master's current role is: LEADER
W20260807 05:08:41.189330 22420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:41.189409 22420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:41.189556 22407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:41.190438 22407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:41.190475 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:41.185646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40259 (local address 127.0.0.1:33559)
0807 05:08:41.185776 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:41.185780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:41.185984 (+ 204us) server_negotiation.cc:407] Connection header received
0807 05:08:41.186094 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:41.186096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:41.186139 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:41.186257 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:41.186559 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:41.187276 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:41.188836 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:41.189074 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:41.189181 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:41.190042 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:41.190045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:41.190047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:41.190059 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:41.190091 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:41.190101 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:41.190195 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:41.190312 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 05:08:41.190343 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:08:41.195812 22407 catalog_manager.cc:1347] Generated new cluster ID: f8271aaddd954c31b7b8a8e443ff831f
I20260807 05:08:41.195879 22407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:41.211576 22407 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:41.212713 22407 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:41.220613 22407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd49f7103b014f1f9a4332e4580566a2: Generated new TSK 0
I20260807 05:08:41.221468 22407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:41.230775 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22324
W20260807 05:08:41.238776 22308 connection.cc:535] client connection to 127.0.0.1:33559 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33559): Connection reset by peer (error 104)
2026-08-07T05:08:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:41 dist-test-slave-1x4g krb5kdc[22428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:41 dist-test-slave-1x4g krb5kdc[22428](info): set up 2 sockets
Aug 07 05:08:41 dist-test-slave-1x4g krb5kdc[22428](info): commencing operation
krb5kdc: starting...
W20260807 05:08:42.281810 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:42 dist-test-slave-1x4g krb5kdc[22428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:42.407240 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:42.517459 22442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:42.517752 22442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:42.517835 22442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:42.521786 22442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:42.521875 22442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:42.521909 22442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:42.521950 22442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:42.521982 22442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:42.527089 22442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34665
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:42.528158 22442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:42.529201 22442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:42.537766 22451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:42.537837 22447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:42.537883 22442 server_base.cc:1034] running on GCE node
W20260807 05:08:42.537921 22448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:42.538611 22442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:42.539659 22442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:42.540879 22442 hybrid_clock.cc:648] HybridClock initialized: now 1786079322540847 us; error 37 us; skew 500 ppm
Aug 07 05:08:42 dist-test-slave-1x4g krb5kdc[22428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:42.543746 22442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:42.544541 22442 webserver.cc:460] Webserver started at http://127.0.0.1:44043/ using document root <none> and password file <none>
I20260807 05:08:42.545120 22442 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:42.545193 22442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:42.545351 22442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:42.547272 22442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "e8ee037111354a08867d0a127364ccea"
format_stamp: "Formatted at 2026-08-07 05:08:42 on dist-test-slave-1x4g"
server_key: "509bfb66949bc347465a5d5283002624"
server_key_iv: "d9f433fe404a50846a47ac499a0a1792"
server_key_version: "encryptionkey@0"
I20260807 05:08:42.547801 22442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "e8ee037111354a08867d0a127364ccea"
format_stamp: "Formatted at 2026-08-07 05:08:42 on dist-test-slave-1x4g"
server_key: "509bfb66949bc347465a5d5283002624"
server_key_iv: "d9f433fe404a50846a47ac499a0a1792"
server_key_version: "encryptionkey@0"
I20260807 05:08:42.551342 22442 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 05:08:42.553761 22458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:42.555075 22442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:42.555184 22442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "e8ee037111354a08867d0a127364ccea"
format_stamp: "Formatted at 2026-08-07 05:08:42 on dist-test-slave-1x4g"
server_key: "509bfb66949bc347465a5d5283002624"
server_key_iv: "d9f433fe404a50846a47ac499a0a1792"
server_key_version: "encryptionkey@0"
I20260807 05:08:42.555270 22442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:42.559653 22442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:42.562656 22442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:42.562803 22442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:42.569653 22442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34665
I20260807 05:08:42.569679 22512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34665 every 8 connection(s)
I20260807 05:08:42.570529 22442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:42.573302 22513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:42.573444 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22442
I20260807 05:08:42.573587 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:42.573904 1095 external_mini_cluster.cc:1428] Setting key 7ab1d14cbeb1e96d6c707778a92a0c0e
I20260807 05:08:42.579164 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:42.575317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51111 (local address 127.0.0.1:34665)
0807 05:08:42.575769 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:42.575784 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:42.575819 (+ 35us) server_negotiation.cc:407] Connection header received
0807 05:08:42.576549 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:42.576567 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:42.576960 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:42.577296 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:42.578049 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:42.578054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:42.578066 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:42.578076 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:42.578126 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:42.578144 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:42.578303 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:42.578369 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 05:08:42.578518 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":122,"threads_started":1}
I20260807 05:08:42.580071 22513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea: Bootstrap starting.
I20260807 05:08:42.581949 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:42.582685 22513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:42.583477 22513 log.cc:824] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:42.585465 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:42.583945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35809 (local address 127.0.0.1:34665)
0807 05:08:42.584072 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:42.584078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:42.584228 (+ 150us) server_negotiation.cc:407] Connection header received
0807 05:08:42.584343 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:42.584345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:42.584369 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:42.584431 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:42.585115 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:42.585118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:42.585119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:42.585129 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:42.585150 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:42.585159 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:42.585223 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:42.585322 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 05:08:42.585348 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:08:42.585796 22513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea: No bootstrap required, opened a new log
I20260807 05:08:42.589252 22513 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ee037111354a08867d0a127364ccea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260807 05:08:42.589489 22513 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:42.589613 22513 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ee037111354a08867d0a127364ccea, State: Initialized, Role: FOLLOWER
I20260807 05:08:42.590030 22513 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ee037111354a08867d0a127364ccea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260807 05:08:42.590181 22513 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:42.590268 22513 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:42.590366 22513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:42.591231 22513 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ee037111354a08867d0a127364ccea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260807 05:08:42.591564 22513 leader_election.cc:302] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8ee037111354a08867d0a127364ccea; no voters:
I20260807 05:08:42.591897 22513 leader_election.cc:288] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:42.591957 22523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:42.592177 22523 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [term 1 LEADER]: Becoming Leader. State: Replica: e8ee037111354a08867d0a127364ccea, State: Running, Role: LEADER
I20260807 05:08:42.592525 22523 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ee037111354a08867d0a127364ccea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } }
I20260807 05:08:42.593191 22513 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:42.594388 22524 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8ee037111354a08867d0a127364ccea. Latest consensus state: current_term: 1 leader_uuid: "e8ee037111354a08867d0a127364ccea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ee037111354a08867d0a127364ccea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } } }
I20260807 05:08:42.594412 22525 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8ee037111354a08867d0a127364ccea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ee037111354a08867d0a127364ccea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34665 } } }
I20260807 05:08:42.594586 22525 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:42.594580 22524 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:42.595594 22533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:42.598227 22533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:42.603529 22533 catalog_manager.cc:1347] Generated new cluster ID: 48ff18d4eb1c4b3ebf5518f06a8906c5
I20260807 05:08:42.603591 22533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:42.613243 22533 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:42.613534 22533 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:42.624573 22533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8ee037111354a08867d0a127364ccea: Generated new TSK 0
I20260807 05:08:42.625378 22533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:42.636742 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22442
2026-08-07T05:08:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:42 dist-test-slave-1x4g krb5kdc[22545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:42 dist-test-slave-1x4g krb5kdc[22545](info): set up 2 sockets
Aug 07 05:08:42 dist-test-slave-1x4g krb5kdc[22545](info): commencing operation
krb5kdc: starting...
W20260807 05:08:43.718304 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:43 dist-test-slave-1x4g krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:43.844205 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46689
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:43.954396 22561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:43.954639 22561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:43.954679 22561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:43.958639 22561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:43.958699 22561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:43.958719 22561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:43.958735 22561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:43.958750 22561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:43.963864 22561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:43.965101 22561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:43.966058 22561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:43.974838 22569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:43.974877 22566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:43.974992 22567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:43.974929 22561 server_base.cc:1034] running on GCE node
I20260807 05:08:43.975630 22561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:43.976720 22561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:43.977960 22561 hybrid_clock.cc:648] HybridClock initialized: now 1786079323977948 us; error 37 us; skew 500 ppm
Aug 07 05:08:43 dist-test-slave-1x4g krb5kdc[22545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:43.980635 22561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:43.981559 22561 webserver.cc:460] Webserver started at http://127.0.0.1:36101/ using document root <none> and password file <none>
I20260807 05:08:43.982158 22561 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:43.982251 22561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:43.982488 22561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:43.984489 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "5194409899584a51a2d3df723f637ec6"
format_stamp: "Formatted at 2026-08-07 05:08:43 on dist-test-slave-1x4g"
server_key: "232cd011322c24ee48f4d966e164f8c4"
server_key_iv: "d6b5c22ec5add06298ecee37ce77cafe"
server_key_version: "encryptionkey@0"
I20260807 05:08:43.985095 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "5194409899584a51a2d3df723f637ec6"
format_stamp: "Formatted at 2026-08-07 05:08:43 on dist-test-slave-1x4g"
server_key: "232cd011322c24ee48f4d966e164f8c4"
server_key_iv: "d6b5c22ec5add06298ecee37ce77cafe"
server_key_version: "encryptionkey@0"
I20260807 05:08:43.988821 22561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:08:43.991370 22578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:43.992534 22561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:43.992669 22561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "5194409899584a51a2d3df723f637ec6"
format_stamp: "Formatted at 2026-08-07 05:08:43 on dist-test-slave-1x4g"
server_key: "232cd011322c24ee48f4d966e164f8c4"
server_key_iv: "d6b5c22ec5add06298ecee37ce77cafe"
server_key_version: "encryptionkey@0"
I20260807 05:08:43.992779 22561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:44.008561 22561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:44.011380 22561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:44.011576 22561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:44.018716 22561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46689
I20260807 05:08:44.018743 22633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46689 every 8 connection(s)
I20260807 05:08:44.019624 22561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:44.020905 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22561
I20260807 05:08:44.021039 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:44.021361 1095 external_mini_cluster.cc:1428] Setting key 0906fa3b18060ec462def34ccb4ed2ee
I20260807 05:08:44.022703 22634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:44.029477 22634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6: Bootstrap starting.
Aug 07 05:08:44 dist-test-slave-1x4g krb5kdc[22545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:08:44.031817 22634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:44.032768 22634 log.cc:824] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:44.034650 22634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6: No bootstrap required, opened a new log
I20260807 05:08:44.036356 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:44.022722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35327 (local address 127.0.0.1:46689)
0807 05:08:44.023177 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:44.023200 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:44.023242 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:08:44.024070 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:44.024093 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:44.024514 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:44.024921 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:44.025621 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:44.027072 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:44.027967 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:44.028725 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:44.028911 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:44.031310 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:44.031316 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:44.031331 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:44.031362 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:44.033388 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:44.033879 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:44.033885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:44.033897 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:44.034007 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:44.034290 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:44.034294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:44.034296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:44.034686 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:44.034883 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:44.035191 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 05:08:44.035472 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":135,"threads_started":1}
I20260807 05:08:44.037981 22634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5194409899584a51a2d3df723f637ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46689 } }
I20260807 05:08:44.038275 22634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:44.038383 22634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5194409899584a51a2d3df723f637ec6, State: Initialized, Role: FOLLOWER
I20260807 05:08:44.038923 22634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5194409899584a51a2d3df723f637ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46689 } }
I20260807 05:08:44.039098 22634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:44.039170 22634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:44.039273 22634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:44.039391 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:44.040374 22634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5194409899584a51a2d3df723f637ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46689 } }
I20260807 05:08:44.040736 22634 leader_election.cc:302] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5194409899584a51a2d3df723f637ec6; no voters:
I20260807 05:08:44.041108 22634 leader_election.cc:288] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:44.041185 22644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:44.041379 22644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [term 1 LEADER]: Becoming Leader. State: Replica: 5194409899584a51a2d3df723f637ec6, State: Running, Role: LEADER
I20260807 05:08:44.041746 22644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5194409899584a51a2d3df723f637ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46689 } }
I20260807 05:08:44.042443 22634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:44.042920 22644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5194409899584a51a2d3df723f637ec6. Latest consensus state: current_term: 1 leader_uuid: "5194409899584a51a2d3df723f637ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5194409899584a51a2d3df723f637ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46689 } } }
I20260807 05:08:44.043515 22645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5194409899584a51a2d3df723f637ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5194409899584a51a2d3df723f637ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46689 } } }
I20260807 05:08:44.043681 22645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:44.043835 22644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:44.044150 22651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:44.048101 22651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:44.051102 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:44.041970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50707 (local address 127.0.0.1:46689)
0807 05:08:44.042085 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:44.042089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:44.042376 (+ 287us) server_negotiation.cc:407] Connection header received
0807 05:08:44.042495 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:44.042498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:44.042541 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:44.042622 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:44.043375 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:44.044311 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:44.045416 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:44.045763 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:44.045854 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:44.047518 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:44.047522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:44.047524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:44.047548 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:44.049269 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:44.049736 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:44.049740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:44.049741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:44.049824 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:44.050150 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:44.050156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:44.050159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:44.050397 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:44.050570 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:44.050819 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 05:08:44.050936 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":43}
I20260807 05:08:44.054291 22651 catalog_manager.cc:1347] Generated new cluster ID: 404b4f8e9da242bf839deecf3cf9effa
I20260807 05:08:44.054390 22651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:44.071035 22651 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:44.071913 22651 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:44.082458 22651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5194409899584a51a2d3df723f637ec6: Generated new TSK 0
I20260807 05:08:44.083132 22651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:44.094797 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22561
2026-08-07T05:08:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:44 dist-test-slave-1x4g krb5kdc[22668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:44 dist-test-slave-1x4g krb5kdc[22668](info): set up 2 sockets
Aug 07 05:08:44 dist-test-slave-1x4g krb5kdc[22668](info): commencing operation
krb5kdc: starting...
W20260807 05:08:45.141811 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:45 dist-test-slave-1x4g krb5kdc[22668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:45.265818 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:45.374892 22681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:45.375135 22681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:45.375174 22681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:45.379108 22681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:45.379158 22681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:45.379177 22681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:45.379194 22681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:45.379208 22681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:45.384335 22681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:45.385720 22681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:45.386770 22681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:08:45.395649 22681 server_base.cc:1034] running on GCE node
W20260807 05:08:45.395594 22690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:45.395577 22688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:45.395602 22687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:45.396330 22681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:45.397560 22681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:45.398752 22681 hybrid_clock.cc:648] HybridClock initialized: now 1786079325398739 us; error 25 us; skew 500 ppm
Aug 07 05:08:45 dist-test-slave-1x4g krb5kdc[22668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079325, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:45.405560 22681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:45.406553 22681 webserver.cc:460] Webserver started at http://127.0.0.1:37689/ using document root <none> and password file <none>
I20260807 05:08:45.407173 22681 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:45.407267 22681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:45.407521 22681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:45.410125 22681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf"
format_stamp: "Formatted at 2026-08-07 05:08:45 on dist-test-slave-1x4g"
server_key: "82d1b6f662aefb110a9e2e8ad6829b15"
server_key_iv: "3e3321bddfa3e672e332e0e66b397123"
server_key_version: "encryptionkey@0"
I20260807 05:08:45.410729 22681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf"
format_stamp: "Formatted at 2026-08-07 05:08:45 on dist-test-slave-1x4g"
server_key: "82d1b6f662aefb110a9e2e8ad6829b15"
server_key_iv: "3e3321bddfa3e672e332e0e66b397123"
server_key_version: "encryptionkey@0"
I20260807 05:08:45.414561 22681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 05:08:45.417001 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:45.418208 22681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:45.418334 22681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf"
format_stamp: "Formatted at 2026-08-07 05:08:45 on dist-test-slave-1x4g"
server_key: "82d1b6f662aefb110a9e2e8ad6829b15"
server_key_iv: "3e3321bddfa3e672e332e0e66b397123"
server_key_version: "encryptionkey@0"
I20260807 05:08:45.418438 22681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:45.437799 22681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:45.441148 22681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:45.441391 22681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:45.448598 22681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38017
I20260807 05:08:45.448676 22753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38017 every 8 connection(s)
I20260807 05:08:45.449582 22681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:45.451527 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22681
I20260807 05:08:45.451673 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:45.451998 1095 external_mini_cluster.cc:1428] Setting key a8fb9cdc4884d13b20b404a0fca8b13f
I20260807 05:08:45.452433 22754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:45.458297 22754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf: Bootstrap starting.
I20260807 05:08:45.460516 22754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:45.460534 22757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:45.453359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51049 (local address 127.0.0.1:38017)
0807 05:08:45.453798 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:45.453812 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:45.453845 (+ 33us) server_negotiation.cc:407] Connection header received
0807 05:08:45.454499 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:45.454515 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:45.454821 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:45.455139 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:45.455534 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:45.456482 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:45.457480 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:45.457958 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:45.458142 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:45.458941 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:45.458946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:45.458958 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:45.459337 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:45.459394 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:45.459422 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:45.459533 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:45.459655 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 05:08:45.459853 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":165,"threads_started":1}
I20260807 05:08:45.461467 22754 log.cc:824] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:45.463347 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:45.463518 22754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf: No bootstrap required, opened a new log
I20260807 05:08:45.467540 22754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38017 } }
I20260807 05:08:45.467818 22754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:45.467917 22754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1434a2e9aecd4f7ea59b4e0ff39a39cf, State: Initialized, Role: FOLLOWER
I20260807 05:08:45.468393 22754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38017 } }
I20260807 05:08:45.468581 22754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:45.468648 22754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:45.468765 22754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:45.469355 22757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:45.465309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58405 (local address 127.0.0.1:38017)
0807 05:08:45.465401 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:45.465404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:45.465611 (+ 207us) server_negotiation.cc:407] Connection header received
0807 05:08:45.465706 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:45.465708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:45.465737 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:45.465803 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:45.466132 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:45.466589 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:45.467698 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:45.468009 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:45.468099 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:45.468916 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:45.468920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:45.468923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:45.468935 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:45.468959 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:45.468971 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:45.469038 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:45.469170 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 05:08:45.469210 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 05:08:45.469897 22754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38017 } }
I20260807 05:08:45.470250 22754 leader_election.cc:302] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1434a2e9aecd4f7ea59b4e0ff39a39cf; no voters:
I20260807 05:08:45.470608 22754 leader_election.cc:288] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:45.470675 22765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:45.470891 22765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [term 1 LEADER]: Becoming Leader. State: Replica: 1434a2e9aecd4f7ea59b4e0ff39a39cf, State: Running, Role: LEADER
I20260807 05:08:45.471220 22765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38017 } }
I20260807 05:08:45.471870 22754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:45.472927 22767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38017 } } }
I20260807 05:08:45.473076 22767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:45.473459 22766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1434a2e9aecd4f7ea59b4e0ff39a39cf. Latest consensus state: current_term: 1 leader_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1434a2e9aecd4f7ea59b4e0ff39a39cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38017 } } }
I20260807 05:08:45.473522 22774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:45.473620 22766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:45.476295 22774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:45.481240 22774 catalog_manager.cc:1347] Generated new cluster ID: 24532e98714946459c1b7aad64602fa7
I20260807 05:08:45.481299 22774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:45.494161 22774 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:45.495348 22774 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:45.500133 22774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1434a2e9aecd4f7ea59b4e0ff39a39cf: Generated new TSK 0
I20260807 05:08:45.500847 22774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:45.511724 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22681
2026-08-07T05:08:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:45 dist-test-slave-1x4g krb5kdc[22787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:45 dist-test-slave-1x4g krb5kdc[22787](info): set up 2 sockets
Aug 07 05:08:45 dist-test-slave-1x4g krb5kdc[22787](info): commencing operation
krb5kdc: starting...
W20260807 05:08:46.562216 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:46 dist-test-slave-1x4g krb5kdc[22787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:46.687248 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:46.798925 22800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:46.799220 22800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:46.799299 22800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:46.803514 22800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:46.803606 22800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:46.803643 22800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:46.803696 22800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:46.803730 22800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:46.808897 22800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:46.809934 22800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:46.811034 22800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:46.819804 22805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:46.819831 22808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:46.819890 22800 server_base.cc:1034] running on GCE node
W20260807 05:08:46.819831 22806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:46.820595 22800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:46.821686 22800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:46.822860 22800 hybrid_clock.cc:648] HybridClock initialized: now 1786079326822839 us; error 33 us; skew 500 ppm
Aug 07 05:08:46 dist-test-slave-1x4g krb5kdc[22787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079326, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:46.825706 22800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:46.826612 22800 webserver.cc:460] Webserver started at http://127.0.0.1:42897/ using document root <none> and password file <none>
I20260807 05:08:46.827221 22800 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:46.827314 22800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:46.827616 22800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:46.830883 22800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "f492e05bb64d40fb93e1a88ffd24ed21"
format_stamp: "Formatted at 2026-08-07 05:08:46 on dist-test-slave-1x4g"
server_key: "c40dd434b988ba0232cafb7377a4b2c3"
server_key_iv: "e06d0eb56511593275b4f31cebac09c6"
server_key_version: "encryptionkey@0"
I20260807 05:08:46.831581 22800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "f492e05bb64d40fb93e1a88ffd24ed21"
format_stamp: "Formatted at 2026-08-07 05:08:46 on dist-test-slave-1x4g"
server_key: "c40dd434b988ba0232cafb7377a4b2c3"
server_key_iv: "e06d0eb56511593275b4f31cebac09c6"
server_key_version: "encryptionkey@0"
I20260807 05:08:46.836737 22800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 05:08:46.839164 22815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:46.840351 22800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:46.840497 22800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "f492e05bb64d40fb93e1a88ffd24ed21"
format_stamp: "Formatted at 2026-08-07 05:08:46 on dist-test-slave-1x4g"
server_key: "c40dd434b988ba0232cafb7377a4b2c3"
server_key_iv: "e06d0eb56511593275b4f31cebac09c6"
server_key_version: "encryptionkey@0"
I20260807 05:08:46.840605 22800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:46.845216 22800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:46.847959 22800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:46.848138 22800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:46.855556 22800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46429
I20260807 05:08:46.855582 22870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46429 every 8 connection(s)
I20260807 05:08:46.856431 22800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:46.859202 22871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:46.863410 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22800
I20260807 05:08:46.863545 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:46.863816 1095 external_mini_cluster.cc:1428] Setting key ee27fe1e93a2902818e0d1595d8e98e9
I20260807 05:08:46.864980 22871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21: Bootstrap starting.
I20260807 05:08:46.867230 22871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:46.868008 22871 log.cc:824] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:46.868999 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:46.865222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36313 (local address 127.0.0.1:46429)
0807 05:08:46.865633 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:46.865648 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:46.865682 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:08:46.866281 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:46.866314 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:46.866579 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:46.866905 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:46.867673 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:46.867678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:46.867689 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:46.867699 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:46.867748 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:46.867766 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:46.867875 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:46.867986 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 05:08:46.868137 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":129,"threads_started":1}
I20260807 05:08:46.869946 22871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21: No bootstrap required, opened a new log
I20260807 05:08:46.871548 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:46.873874 22871 raft_consensus.cc:348] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 05:08:46.874166 22871 raft_consensus.cc:374] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:46.874246 22871 raft_consensus.cc:729] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f492e05bb64d40fb93e1a88ffd24ed21, State: Initialized, Role: FOLLOWER
I20260807 05:08:46.874678 22871 consensus_queue.cc:260] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 05:08:46.874817 22871 raft_consensus.cc:388] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:46.874884 22871 raft_consensus.cc:482] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:46.875007 22871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:46.875913 22871 raft_consensus.cc:504] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 05:08:46.876247 22871 leader_election.cc:302] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f492e05bb64d40fb93e1a88ffd24ed21; no voters:
I20260807 05:08:46.876538 22871 leader_election.cc:288] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:46.876714 22883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:46.876956 22883 raft_consensus.cc:686] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [term 1 LEADER]: Becoming Leader. State: Replica: f492e05bb64d40fb93e1a88ffd24ed21, State: Running, Role: LEADER
I20260807 05:08:46.877334 22883 consensus_queue.cc:237] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } }
I20260807 05:08:46.877846 22871 sys_catalog.cc:564] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:46.878536 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:46.873618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59815 (local address 127.0.0.1:46429)
0807 05:08:46.873717 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:46.873719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:46.876987 (+ 3268us) server_negotiation.cc:407] Connection header received
0807 05:08:46.877190 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:46.877193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:46.877226 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:46.877273 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:46.878122 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:46.878126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:46.878128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:46.878139 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:46.878166 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:46.878182 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:46.878281 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:46.878387 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 05:08:46.878423 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 05:08:46.879077 22884 sys_catalog.cc:455] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f492e05bb64d40fb93e1a88ffd24ed21. Latest consensus state: current_term: 1 leader_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } } }
I20260807 05:08:46.879196 22884 sys_catalog.cc:458] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:46.879424 22885 sys_catalog.cc:455] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f492e05bb64d40fb93e1a88ffd24ed21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46429 } } }
I20260807 05:08:46.879817 22885 sys_catalog.cc:458] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:46.880091 22894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:46.883123 22894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:46.888098 22894 catalog_manager.cc:1347] Generated new cluster ID: 4ceda2e343a348b081f92ea5b70a108c
I20260807 05:08:46.888178 22894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:46.898321 22894 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:46.898636 22894 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:46.907325 22894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f492e05bb64d40fb93e1a88ffd24ed21: Generated new TSK 0
I20260807 05:08:46.908079 22894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:46.914834 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22800
2026-08-07T05:08:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:46 dist-test-slave-1x4g krb5kdc[22904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:46 dist-test-slave-1x4g krb5kdc[22904](info): set up 2 sockets
Aug 07 05:08:46 dist-test-slave-1x4g krb5kdc[22904](info): commencing operation
krb5kdc: starting...
W20260807 05:08:47.964880 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:48 dist-test-slave-1x4g krb5kdc[22904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:48.089740 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:48.199054 22917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:48.199326 22917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:48.199400 22917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:48.203388 22917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:48.203469 22917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:48.203503 22917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:48.203560 22917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:48.203599 22917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:48.208886 22917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:48.209901 22917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:48.210925 22917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:48.219455 22922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:48.219547 22917 server_base.cc:1034] running on GCE node
W20260807 05:08:48.219491 22925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:48.219435 22923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:48.220152 22917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:48.221235 22917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:48.222400 22917 hybrid_clock.cc:648] HybridClock initialized: now 1786079328222380 us; error 29 us; skew 500 ppm
Aug 07 05:08:48 dist-test-slave-1x4g krb5kdc[22904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079328, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:48.225137 22917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:48.225924 22917 webserver.cc:460] Webserver started at http://127.0.0.1:32863/ using document root <none> and password file <none>
I20260807 05:08:48.226400 22917 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:48.226455 22917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:48.226603 22917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:48.228487 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "257bc157ebdb42919b96696a7fa785e6"
format_stamp: "Formatted at 2026-08-07 05:08:48 on dist-test-slave-1x4g"
server_key: "e8075a773165aa0a4510454ad51ae9f9"
server_key_iv: "9d727fe241ccf63743b3b9111adebc82"
server_key_version: "encryptionkey@0"
I20260807 05:08:48.229094 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "257bc157ebdb42919b96696a7fa785e6"
format_stamp: "Formatted at 2026-08-07 05:08:48 on dist-test-slave-1x4g"
server_key: "e8075a773165aa0a4510454ad51ae9f9"
server_key_iv: "9d727fe241ccf63743b3b9111adebc82"
server_key_version: "encryptionkey@0"
I20260807 05:08:48.232583 22917 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:08:48.235008 22932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:48.236184 22917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:48.236321 22917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "257bc157ebdb42919b96696a7fa785e6"
format_stamp: "Formatted at 2026-08-07 05:08:48 on dist-test-slave-1x4g"
server_key: "e8075a773165aa0a4510454ad51ae9f9"
server_key_iv: "9d727fe241ccf63743b3b9111adebc82"
server_key_version: "encryptionkey@0"
I20260807 05:08:48.236433 22917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:48.254709 22917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:48.257437 22917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:48.257639 22917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:48.265254 22917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33021
I20260807 05:08:48.265326 22987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33021 every 8 connection(s)
I20260807 05:08:48.266332 22917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:48.269002 22988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:48.274240 22988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6: Bootstrap starting.
I20260807 05:08:48.275635 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 22917
I20260807 05:08:48.275767 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:48.276078 1095 external_mini_cluster.cc:1428] Setting key c22d705d1b4f80206f3a6f60ff30c3d3
I20260807 05:08:48.276815 22988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:48.277608 22988 log.cc:824] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:48.279531 22988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6: No bootstrap required, opened a new log
I20260807 05:08:48.283493 22988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257bc157ebdb42919b96696a7fa785e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260807 05:08:48.283757 22988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:48.283825 22988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257bc157ebdb42919b96696a7fa785e6, State: Initialized, Role: FOLLOWER
I20260807 05:08:48.284334 22988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257bc157ebdb42919b96696a7fa785e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260807 05:08:48.284510 22988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:48.284577 22988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:48.284699 22988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 05:08:48 dist-test-slave-1x4g krb5kdc[22904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:08:48.285925 22988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257bc157ebdb42919b96696a7fa785e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260807 05:08:48.286276 22988 leader_election.cc:302] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 257bc157ebdb42919b96696a7fa785e6; no voters:
I20260807 05:08:48.286661 22988 leader_election.cc:288] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:48.286814 22993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:48.287093 22993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [term 1 LEADER]: Becoming Leader. State: Replica: 257bc157ebdb42919b96696a7fa785e6, State: Running, Role: LEADER
I20260807 05:08:48.287442 22993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257bc157ebdb42919b96696a7fa785e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260807 05:08:48.287948 22988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:48.289304 22994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "257bc157ebdb42919b96696a7fa785e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257bc157ebdb42919b96696a7fa785e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } } }
I20260807 05:08:48.289445 22994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:48.289781 22995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 257bc157ebdb42919b96696a7fa785e6. Latest consensus state: current_term: 1 leader_uuid: "257bc157ebdb42919b96696a7fa785e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257bc157ebdb42919b96696a7fa785e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } } }
I20260807 05:08:48.289932 22995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:48.292011 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:48.277341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:40985 (local address 127.0.0.1:33021)
0807 05:08:48.277854 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:48.277871 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:48.277908 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:08:48.278696 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:48.278715 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:48.279044 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:48.279324 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:48.280157 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:48.281113 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:48.282445 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:48.282945 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:48.283075 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:48.286120 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:48.286129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:48.286150 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:48.286178 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:48.288696 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:48.289464 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:48.289470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:48.289476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:48.289627 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:48.290097 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:48.290100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:48.290101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:48.290622 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:48.290866 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:48.291124 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 05:08:48.291332 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":165,"threads_started":1}
W20260807 05:08:48.292580 23008 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:48.292644 23008 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:48.293152 23005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:48.293982 23005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:48.295266 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:48.299324 23005 catalog_manager.cc:1347] Generated new cluster ID: 252d3cacbe794d94aa098a49ed5adfcd
I20260807 05:08:48.299379 23005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:48.307613 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:48.297584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:51089 (local address 127.0.0.1:33021)
0807 05:08:48.297686 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:48.297690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:48.297945 (+ 255us) server_negotiation.cc:407] Connection header received
0807 05:08:48.298057 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:48.298060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:48.298104 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:48.298184 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:48.299076 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:48.300218 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:48.301489 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:48.301850 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:48.301938 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:48.303757 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:48.303761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:48.303764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:48.303788 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:48.305328 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:48.306070 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:48.306074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:48.306076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:48.306166 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:48.306597 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:48.306600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:48.306602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:48.306799 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:48.306910 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:48.307343 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 05:08:48.307449 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:08:48.316196 23005 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:48.317124 23005 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:48.323879 23005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 257bc157ebdb42919b96696a7fa785e6: Generated new TSK 0
I20260807 05:08:48.324532 23005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:48.333582 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 22917
W20260807 05:08:48.341612 22902 connection.cc:535] client connection to 127.0.0.1:33021 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33021): Connection reset by peer (error 104)
2026-08-07T05:08:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:48 dist-test-slave-1x4g krb5kdc[23020](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:48 dist-test-slave-1x4g krb5kdc[23020](info): set up 2 sockets
Aug 07 05:08:48 dist-test-slave-1x4g krb5kdc[23020](info): commencing operation
krb5kdc: starting...
W20260807 05:08:49.386512 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:49 dist-test-slave-1x4g krb5kdc[23020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:49.507649 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:49.617264 23033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:49.617522 23033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:49.617614 23033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:49.621438 23033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:49.621522 23033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:49.621584 23033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:49.621637 23033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:49.621673 23033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:49.626616 23033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:49.627574 23033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:49.628510 23033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:49.636811 23038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:49.636837 23041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:49.637005 23033 server_base.cc:1034] running on GCE node
W20260807 05:08:49.636826 23039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:49.637625 23033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:49.638707 23033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:49.639886 23033 hybrid_clock.cc:648] HybridClock initialized: now 1786079329639866 us; error 42 us; skew 500 ppm
Aug 07 05:08:49 dist-test-slave-1x4g krb5kdc[23020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:49.642581 23033 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:49.643486 23033 webserver.cc:460] Webserver started at http://127.0.0.1:34199/ using document root <none> and password file <none>
I20260807 05:08:49.644035 23033 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:49.644122 23033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:49.644348 23033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:49.646311 23033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "69fa4e9ca0e24dfcb0485ca200584598"
format_stamp: "Formatted at 2026-08-07 05:08:49 on dist-test-slave-1x4g"
server_key: "89771b03aa599860e0277aa2acc70025"
server_key_iv: "5bf96a6fa408f373954ccb35ffb41398"
server_key_version: "encryptionkey@0"
I20260807 05:08:49.646852 23033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "69fa4e9ca0e24dfcb0485ca200584598"
format_stamp: "Formatted at 2026-08-07 05:08:49 on dist-test-slave-1x4g"
server_key: "89771b03aa599860e0277aa2acc70025"
server_key_iv: "5bf96a6fa408f373954ccb35ffb41398"
server_key_version: "encryptionkey@0"
I20260807 05:08:49.650403 23033 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:08:49.652751 23050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:49.653956 23033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:49.654089 23033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "69fa4e9ca0e24dfcb0485ca200584598"
format_stamp: "Formatted at 2026-08-07 05:08:49 on dist-test-slave-1x4g"
server_key: "89771b03aa599860e0277aa2acc70025"
server_key_iv: "5bf96a6fa408f373954ccb35ffb41398"
server_key_version: "encryptionkey@0"
I20260807 05:08:49.654196 23033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:49.679531 23033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:49.682896 23033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:49.683105 23033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:49.690363 23033 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42099
I20260807 05:08:49.690389 23103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42099 every 8 connection(s)
I20260807 05:08:49.691311 23033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:49.693756 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23033
I20260807 05:08:49.693899 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:49.694128 23104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:49.694185 1095 external_mini_cluster.cc:1428] Setting key a35d31298073b24aca0d508886ed2a0f
I20260807 05:08:49.701555 23104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598: Bootstrap starting.
I20260807 05:08:49.704162 23104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:49.704428 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:49.695646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:52311 (local address 127.0.0.1:42099)
0807 05:08:49.696086 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:49.696106 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:49.696149 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:08:49.697109 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:49.697134 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:49.697587 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:49.697963 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:49.698372 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:49.699842 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:49.700700 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:49.701405 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:49.701585 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:49.702435 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:49.702441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:49.702456 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:49.702964 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:49.703031 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:49.703061 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:49.703191 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:49.703360 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 05:08:49.703495 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260807 05:08:49.705075 23104 log.cc:824] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:49.707008 23104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598: No bootstrap required, opened a new log
I20260807 05:08:49.707294 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:49.711805 23104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fa4e9ca0e24dfcb0485ca200584598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260807 05:08:49.712092 23104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:49.712159 23104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69fa4e9ca0e24dfcb0485ca200584598, State: Initialized, Role: FOLLOWER
I20260807 05:08:49.712603 23104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fa4e9ca0e24dfcb0485ca200584598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260807 05:08:49.712798 23104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:49.712885 23104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:49.712990 23104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:49.713920 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:49.709740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:60833 (local address 127.0.0.1:42099)
0807 05:08:49.709850 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:49.709854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:49.710018 (+ 164us) server_negotiation.cc:407] Connection header received
0807 05:08:49.710139 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:49.710142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:49.710178 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:49.710234 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:49.710515 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:49.710969 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:49.712057 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:49.712412 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:49.712520 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:49.713415 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:49.713418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:49.713420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:49.713430 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:49.713462 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:49.713472 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:49.713538 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:49.713726 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 05:08:49.713778 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:08:49.714128 23104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fa4e9ca0e24dfcb0485ca200584598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260807 05:08:49.714486 23104 leader_election.cc:302] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69fa4e9ca0e24dfcb0485ca200584598; no voters:
I20260807 05:08:49.714823 23104 leader_election.cc:288] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:49.714893 23114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:49.715145 23114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [term 1 LEADER]: Becoming Leader. State: Replica: 69fa4e9ca0e24dfcb0485ca200584598, State: Running, Role: LEADER
I20260807 05:08:49.715546 23114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fa4e9ca0e24dfcb0485ca200584598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260807 05:08:49.716292 23104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:49.717303 23115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69fa4e9ca0e24dfcb0485ca200584598. Latest consensus state: current_term: 1 leader_uuid: "69fa4e9ca0e24dfcb0485ca200584598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fa4e9ca0e24dfcb0485ca200584598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } } }
I20260807 05:08:49.717444 23115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:49.717826 23123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:49.717819 23116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69fa4e9ca0e24dfcb0485ca200584598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69fa4e9ca0e24dfcb0485ca200584598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } } }
I20260807 05:08:49.718055 23116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:49.720968 23123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:49.726894 23123 catalog_manager.cc:1347] Generated new cluster ID: 28c145130c4b40d0994dd703263e3205
I20260807 05:08:49.726955 23123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:49.740600 23123 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:49.741482 23123 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:49.746631 23123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69fa4e9ca0e24dfcb0485ca200584598: Generated new TSK 0
I20260807 05:08:49.747215 23123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:49.750375 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23033
W20260807 05:08:49.758392 23018 connection.cc:535] client connection to 127.0.0.1:42099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42099): Connection reset by peer (error 104)
2026-08-07T05:08:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:49 dist-test-slave-1x4g krb5kdc[23138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:49 dist-test-slave-1x4g krb5kdc[23138](info): set up 2 sockets
Aug 07 05:08:49 dist-test-slave-1x4g krb5kdc[23138](info): commencing operation
krb5kdc: starting...
W20260807 05:08:50.815415 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:50 dist-test-slave-1x4g krb5kdc[23138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:50.948155 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:51.058884 23151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:51.059185 23151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:51.059269 23151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:51.063266 23151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:51.063357 23151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:51.063397 23151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:51.063464 23151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:51.063517 23151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:51.068640 23151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:51.069767 23151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:51.070794 23151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:08:51.079375 23151 server_base.cc:1034] running on GCE node
W20260807 05:08:51.079319 23160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:51.079322 23157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:51.079365 23158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:51.080140 23151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:51.081264 23151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:51.082470 23151 hybrid_clock.cc:648] HybridClock initialized: now 1786079331082450 us; error 30 us; skew 500 ppm
Aug 07 05:08:51 dist-test-slave-1x4g krb5kdc[23138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079331, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:51.090355 23151 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:51.091329 23151 webserver.cc:460] Webserver started at http://127.0.0.1:40283/ using document root <none> and password file <none>
I20260807 05:08:51.091912 23151 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:51.092010 23151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:51.092267 23151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:51.094444 23151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "333378c946d94cfe8c6b9f5fb03f0179"
format_stamp: "Formatted at 2026-08-07 05:08:51 on dist-test-slave-1x4g"
server_key: "8bfa251213b187f937ad403d3de3c88c"
server_key_iv: "308e1b7e7a62cb5c5b7e9ebdf71ecbe2"
server_key_version: "encryptionkey@0"
I20260807 05:08:51.095027 23151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "333378c946d94cfe8c6b9f5fb03f0179"
format_stamp: "Formatted at 2026-08-07 05:08:51 on dist-test-slave-1x4g"
server_key: "8bfa251213b187f937ad403d3de3c88c"
server_key_iv: "308e1b7e7a62cb5c5b7e9ebdf71ecbe2"
server_key_version: "encryptionkey@0"
I20260807 05:08:51.098832 23151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:08:51.101300 23169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:51.102466 23151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:51.102617 23151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "333378c946d94cfe8c6b9f5fb03f0179"
format_stamp: "Formatted at 2026-08-07 05:08:51 on dist-test-slave-1x4g"
server_key: "8bfa251213b187f937ad403d3de3c88c"
server_key_iv: "308e1b7e7a62cb5c5b7e9ebdf71ecbe2"
server_key_version: "encryptionkey@0"
I20260807 05:08:51.102730 23151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:51.107307 23151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:51.110553 23151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:51.110750 23151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:51.118273 23151 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35417
I20260807 05:08:51.118315 23222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35417 every 8 connection(s)
I20260807 05:08:51.119269 23151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:51.122023 23223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:51.124624 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23151
I20260807 05:08:51.124765 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:51.125118 1095 external_mini_cluster.cc:1428] Setting key a1d00f38399badd31d876a1717c9e2a6
I20260807 05:08:51.128278 23223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179: Bootstrap starting.
I20260807 05:08:51.130313 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:51.126560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:51847 (local address 127.0.0.1:35417)
0807 05:08:51.127011 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:51.127027 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:51.127064 (+ 37us) server_negotiation.cc:407] Connection header received
0807 05:08:51.127725 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:51.127742 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:51.128024 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:51.128364 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:51.129187 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:51.129192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:51.129204 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:51.129213 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:51.129264 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:51.129283 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:51.129431 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:51.129509 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 05:08:51.129664 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":131,"threads_started":1}
I20260807 05:08:51.130882 23223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:51.131667 23223 log.cc:824] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:51.133297 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:51.133853 23223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179: No bootstrap required, opened a new log
I20260807 05:08:51.137239 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:51.135372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:36519 (local address 127.0.0.1:35417)
0807 05:08:51.135495 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:51.135499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:51.135705 (+ 206us) server_negotiation.cc:407] Connection header received
0807 05:08:51.135816 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:51.135820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:51.135855 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:51.135936 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:51.136757 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:51.136761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:51.136764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:51.136803 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:51.136834 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:51.136852 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:51.136966 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:51.137063 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 05:08:51.137108 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 05:08:51.137954 23223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333378c946d94cfe8c6b9f5fb03f0179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 05:08:51.138206 23223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:51.138276 23223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333378c946d94cfe8c6b9f5fb03f0179, State: Initialized, Role: FOLLOWER
I20260807 05:08:51.138679 23223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333378c946d94cfe8c6b9f5fb03f0179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 05:08:51.138803 23223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:51.138847 23223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:51.138917 23223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:51.139890 23223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333378c946d94cfe8c6b9f5fb03f0179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 05:08:51.140242 23223 leader_election.cc:302] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 333378c946d94cfe8c6b9f5fb03f0179; no voters:
I20260807 05:08:51.140544 23223 leader_election.cc:288] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:51.140679 23233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:51.140931 23233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [term 1 LEADER]: Becoming Leader. State: Replica: 333378c946d94cfe8c6b9f5fb03f0179, State: Running, Role: LEADER
I20260807 05:08:51.141283 23233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333378c946d94cfe8c6b9f5fb03f0179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 05:08:51.141724 23223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:51.143024 23235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 333378c946d94cfe8c6b9f5fb03f0179. Latest consensus state: current_term: 1 leader_uuid: "333378c946d94cfe8c6b9f5fb03f0179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333378c946d94cfe8c6b9f5fb03f0179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } } }
I20260807 05:08:51.143018 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "333378c946d94cfe8c6b9f5fb03f0179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333378c946d94cfe8c6b9f5fb03f0179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } } }
I20260807 05:08:51.143191 23235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:51.143251 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179 [sys.catalog]: This master's current role is: LEADER
W20260807 05:08:51.146524 23249 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:51.146667 23249 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:51.146768 23245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:51.147469 23245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:51.152585 23245 catalog_manager.cc:1347] Generated new cluster ID: 68f0c5b0fcfd4c7a95d18a39eb809cbf
I20260807 05:08:51.152652 23245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:51.187145 23245 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:51.187501 23245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:51.194583 23245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 333378c946d94cfe8c6b9f5fb03f0179: Generated new TSK 0
I20260807 05:08:51.195338 23245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:51.200505 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23151
2026-08-07T05:08:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:51 dist-test-slave-1x4g krb5kdc[23255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:51 dist-test-slave-1x4g krb5kdc[23255](info): set up 2 sockets
Aug 07 05:08:51 dist-test-slave-1x4g krb5kdc[23255](info): commencing operation
krb5kdc: starting...
W20260807 05:08:52.249517 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:52 dist-test-slave-1x4g krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:52.369982 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:52.478507 23268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:52.478775 23268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:52.478860 23268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:52.482744 23268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:52.482832 23268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:52.482870 23268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:52.482936 23268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:52.482977 23268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:52.487836 23268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38887
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:52.488817 23268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:52.489789 23268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:52.497859 23273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:52.497866 23274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:52.497877 23276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:52.498157 23268 server_base.cc:1034] running on GCE node
I20260807 05:08:52.498701 23268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:52.499742 23268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:52.500945 23268 hybrid_clock.cc:648] HybridClock initialized: now 1786079332500920 us; error 37 us; skew 500 ppm
Aug 07 05:08:52 dist-test-slave-1x4g krb5kdc[23255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079332, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:52.503638 23268 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:52.504515 23268 webserver.cc:460] Webserver started at http://127.0.0.1:39053/ using document root <none> and password file <none>
I20260807 05:08:52.505157 23268 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:52.505262 23268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:52.505515 23268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:52.507433 23268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "4eb019fb8c4440c9923306facbffe338"
format_stamp: "Formatted at 2026-08-07 05:08:52 on dist-test-slave-1x4g"
server_key: "1ec471bd85529ab73b6ecb22f833fe5b"
server_key_iv: "f9dad1517c6aed8b854f453315053ea7"
server_key_version: "encryptionkey@0"
I20260807 05:08:52.507982 23268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "4eb019fb8c4440c9923306facbffe338"
format_stamp: "Formatted at 2026-08-07 05:08:52 on dist-test-slave-1x4g"
server_key: "1ec471bd85529ab73b6ecb22f833fe5b"
server_key_iv: "f9dad1517c6aed8b854f453315053ea7"
server_key_version: "encryptionkey@0"
I20260807 05:08:52.511701 23268 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260807 05:08:52.514148 23286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:52.515295 23268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:08:52.515437 23268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "4eb019fb8c4440c9923306facbffe338"
format_stamp: "Formatted at 2026-08-07 05:08:52 on dist-test-slave-1x4g"
server_key: "1ec471bd85529ab73b6ecb22f833fe5b"
server_key_iv: "f9dad1517c6aed8b854f453315053ea7"
server_key_version: "encryptionkey@0"
I20260807 05:08:52.515547 23268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:52.525229 23268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:52.527781 23268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:52.527968 23268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:52.535569 23268 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38887
I20260807 05:08:52.535606 23341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38887 every 8 connection(s)
I20260807 05:08:52.536510 23268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:52.539518 23342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:52.545024 23342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338: Bootstrap starting.
I20260807 05:08:52.545717 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23268
I20260807 05:08:52.545851 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:52.546131 1095 external_mini_cluster.cc:1428] Setting key 34ee5b97af78b09d1144e108d219d471
I20260807 05:08:52.547858 23342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:52.548542 23342 log.cc:824] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:52.550334 23342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338: No bootstrap required, opened a new log
I20260807 05:08:52.553561 23342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb019fb8c4440c9923306facbffe338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38887 } }
I20260807 05:08:52.553819 23342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:52.553897 23342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eb019fb8c4440c9923306facbffe338, State: Initialized, Role: FOLLOWER
I20260807 05:08:52.554425 23342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb019fb8c4440c9923306facbffe338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38887 } }
I20260807 05:08:52.554598 23342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:52.554664 23342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 07 05:08:52 dist-test-slave-1x4g krb5kdc[23255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 05:08:52.554766 23342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:52.555781 23342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb019fb8c4440c9923306facbffe338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38887 } }
I20260807 05:08:52.556173 23342 leader_election.cc:302] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eb019fb8c4440c9923306facbffe338; no voters:
I20260807 05:08:52.556540 23342 leader_election.cc:288] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:52.556677 23348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:52.556974 23348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [term 1 LEADER]: Becoming Leader. State: Replica: 4eb019fb8c4440c9923306facbffe338, State: Running, Role: LEADER
I20260807 05:08:52.557337 23348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb019fb8c4440c9923306facbffe338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38887 } }
I20260807 05:08:52.557868 23342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:52.558581 23348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eb019fb8c4440c9923306facbffe338. Latest consensus state: current_term: 1 leader_uuid: "4eb019fb8c4440c9923306facbffe338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb019fb8c4440c9923306facbffe338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38887 } } }
I20260807 05:08:52.558769 23348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:52.559029 23349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eb019fb8c4440c9923306facbffe338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb019fb8c4440c9923306facbffe338" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38887 } } }
I20260807 05:08:52.559175 23349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:52.559238 23357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:52.561618 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:52.547351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:43135 (local address 127.0.0.1:38887)
0807 05:08:52.547774 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:52.547788 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:52.547821 (+ 33us) server_negotiation.cc:407] Connection header received
0807 05:08:52.548476 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:52.548492 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:52.548903 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:52.549235 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:52.550046 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:52.551001 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:52.551823 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:52.552314 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:52.552495 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:52.555344 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:52.555350 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:52.555367 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:52.555396 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:52.557521 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:52.558087 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:52.558093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:52.558099 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:52.558241 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:52.558730 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:52.558734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:52.558738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:52.559256 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:52.560228 (+ 972us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:52.560404 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 05:08:52.560648 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":114,"threads_started":1}
I20260807 05:08:52.562649 23357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:52.565065 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:52.568209 23357 catalog_manager.cc:1347] Generated new cluster ID: d1eec27c88404efaad916c5040d33391
I20260807 05:08:52.568267 23357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:52.577373 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:52.567284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:49861 (local address 127.0.0.1:38887)
0807 05:08:52.567376 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:52.567382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:52.569250 (+ 1868us) server_negotiation.cc:407] Connection header received
0807 05:08:52.569348 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:52.569352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:52.569397 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:52.569490 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:52.570179 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:52.570995 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:52.572043 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:52.572451 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:52.572554 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:52.574084 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:52.574088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:52.574090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:52.574113 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:52.575617 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:52.576083 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:52.576086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:52.576088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:52.576164 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:52.576476 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:52.576479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:52.576481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:52.576696 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:52.576875 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:52.577129 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 05:08:52.577223 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 05:08:52.590648 23357 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:52.591692 23357 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:52.600545 23357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eb019fb8c4440c9923306facbffe338: Generated new TSK 0
I20260807 05:08:52.601218 23357 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:52.610428 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23268
W20260807 05:08:52.618404 23253 connection.cc:535] client connection to 127.0.0.1:38887 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38887): Connection reset by peer (error 104)
2026-08-07T05:08:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:52 dist-test-slave-1x4g krb5kdc[23375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:52 dist-test-slave-1x4g krb5kdc[23375](info): set up 2 sockets
Aug 07 05:08:52 dist-test-slave-1x4g krb5kdc[23375](info): commencing operation
krb5kdc: starting...
W20260807 05:08:53.669577 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:53 dist-test-slave-1x4g krb5kdc[23375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:53.793409 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:53.910801 23388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:53.911032 23388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:53.911070 23388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:53.914867 23388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:53.914932 23388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:53.914951 23388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:53.914968 23388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:53.914983 23388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:53.920117 23388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:53.921056 23388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:53.921969 23388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:53.930321 23393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:53.930370 23388 server_base.cc:1034] running on GCE node
W20260807 05:08:53.930359 23396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:53.930311 23394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:53.931149 23388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:53.932245 23388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:53.933416 23388 hybrid_clock.cc:648] HybridClock initialized: now 1786079333933398 us; error 27 us; skew 500 ppm
Aug 07 05:08:53 dist-test-slave-1x4g krb5kdc[23375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079333, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:53.936074 23388 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:53.937026 23388 webserver.cc:460] Webserver started at http://127.0.0.1:34021/ using document root <none> and password file <none>
I20260807 05:08:53.937577 23388 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:53.937665 23388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:53.937898 23388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:53.939929 23388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "aea85794e1d3488d850c297096c2516e"
format_stamp: "Formatted at 2026-08-07 05:08:53 on dist-test-slave-1x4g"
server_key: "5d1efbd90721ace73cc1ff91d7cd4eed"
server_key_iv: "39aeaa696a431c0b93acb7eb18796306"
server_key_version: "encryptionkey@0"
I20260807 05:08:53.940503 23388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "aea85794e1d3488d850c297096c2516e"
format_stamp: "Formatted at 2026-08-07 05:08:53 on dist-test-slave-1x4g"
server_key: "5d1efbd90721ace73cc1ff91d7cd4eed"
server_key_iv: "39aeaa696a431c0b93acb7eb18796306"
server_key_version: "encryptionkey@0"
I20260807 05:08:53.944149 23388 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 05:08:53.946568 23403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:53.947737 23388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 05:08:53.947885 23388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "aea85794e1d3488d850c297096c2516e"
format_stamp: "Formatted at 2026-08-07 05:08:53 on dist-test-slave-1x4g"
server_key: "5d1efbd90721ace73cc1ff91d7cd4eed"
server_key_iv: "39aeaa696a431c0b93acb7eb18796306"
server_key_version: "encryptionkey@0"
I20260807 05:08:53.948010 23388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:53.957531 23388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:53.960204 23388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:53.960402 23388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:53.967612 23388 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36523
I20260807 05:08:53.967670 23457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36523 every 8 connection(s)
I20260807 05:08:53.968561 23388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:53.969024 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23388
I20260807 05:08:53.969170 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:53.969468 1095 external_mini_cluster.cc:1428] Setting key 7734d1f32d0b86cd16ebd5bbfde764c7
I20260807 05:08:53.971701 23458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:53.978211 23458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e: Bootstrap starting.
I20260807 05:08:53.978488 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:53.970828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:34165 (local address 127.0.0.1:36523)
0807 05:08:53.971248 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:53.971263 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:53.971317 (+ 54us) server_negotiation.cc:407] Connection header received
0807 05:08:53.972044 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:53.972062 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:53.972389 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:53.972658 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:53.973033 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:53.973996 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:53.975238 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:53.975690 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:53.975800 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:53.976708 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:53.976713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:53.976725 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:53.977163 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:53.977215 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:53.977238 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:53.977338 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:53.977583 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 05:08:53.977728 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":142,"threads_started":1}
I20260807 05:08:53.980747 23458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:53.981554 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:53.981701 23458 log.cc:824] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:53.983726 23458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e: No bootstrap required, opened a new log
I20260807 05:08:53.988018 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:53.983771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:53521 (local address 127.0.0.1:36523)
0807 05:08:53.983925 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:53.983931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:53.984117 (+ 186us) server_negotiation.cc:407] Connection header received
0807 05:08:53.984243 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:53.984246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:53.984289 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:53.984363 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:53.984657 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:53.985205 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:53.986393 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:53.986599 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:53.986671 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:53.987527 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:53.987530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:53.987532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:53.987543 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:53.987566 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:53.987576 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:53.987643 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:53.987809 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 05:08:53.987840 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 05:08:53.988528 23458 raft_consensus.cc:348] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea85794e1d3488d850c297096c2516e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260807 05:08:53.988760 23458 raft_consensus.cc:374] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:53.988862 23458 raft_consensus.cc:729] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aea85794e1d3488d850c297096c2516e, State: Initialized, Role: FOLLOWER
I20260807 05:08:53.989331 23458 consensus_queue.cc:260] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea85794e1d3488d850c297096c2516e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260807 05:08:53.989514 23458 raft_consensus.cc:388] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:53.989586 23458 raft_consensus.cc:482] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:53.989718 23458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:53.990660 23458 raft_consensus.cc:504] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea85794e1d3488d850c297096c2516e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260807 05:08:53.991031 23458 leader_election.cc:302] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aea85794e1d3488d850c297096c2516e; no voters:
I20260807 05:08:53.991343 23458 leader_election.cc:288] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:53.991469 23468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:53.991729 23468 raft_consensus.cc:686] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [term 1 LEADER]: Becoming Leader. State: Replica: aea85794e1d3488d850c297096c2516e, State: Running, Role: LEADER
I20260807 05:08:53.992089 23468 consensus_queue.cc:237] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea85794e1d3488d850c297096c2516e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } }
I20260807 05:08:53.992676 23458 sys_catalog.cc:564] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:53.993923 23471 sys_catalog.cc:455] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [sys.catalog]: SysCatalogTable state changed. Reason: New leader aea85794e1d3488d850c297096c2516e. Latest consensus state: current_term: 1 leader_uuid: "aea85794e1d3488d850c297096c2516e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea85794e1d3488d850c297096c2516e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } } }
I20260807 05:08:53.993953 23469 sys_catalog.cc:455] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aea85794e1d3488d850c297096c2516e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea85794e1d3488d850c297096c2516e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36523 } } }
I20260807 05:08:53.994076 23471 sys_catalog.cc:458] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:53.994078 23469 sys_catalog.cc:458] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:53.995142 23478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:53.998140 23478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:54.003933 23478 catalog_manager.cc:1347] Generated new cluster ID: 74876c6d685d49a29d11307d5997b1ed
I20260807 05:08:54.004063 23478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:54.017503 23478 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:54.018396 23478 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:54.027513 23478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aea85794e1d3488d850c297096c2516e: Generated new TSK 0
I20260807 05:08:54.028151 23478 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:54.038522 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23388
W20260807 05:08:54.046034 23373 connection.cc:535] client connection to 127.0.0.1:36523 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36523): Connection reset by peer (error 104)
2026-08-07T05:08:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:54 dist-test-slave-1x4g krb5kdc[23491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:54 dist-test-slave-1x4g krb5kdc[23491](info): set up 2 sockets
Aug 07 05:08:54 dist-test-slave-1x4g krb5kdc[23491](info): commencing operation
krb5kdc: starting...
W20260807 05:08:55.097764 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:55 dist-test-slave-1x4g krb5kdc[23491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 05:08:55.223821 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:55.335165 23504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:55.335409 23504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:55.335449 23504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:55.339323 23504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:55.339381 23504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:55.339401 23504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:55.339416 23504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:55.339432 23504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:55.344635 23504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:55.345681 23504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:55.346671 23504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:55.355386 23511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:55.355479 23504 server_base.cc:1034] running on GCE node
W20260807 05:08:55.355410 23510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:55.355396 23513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:55.356168 23504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:55.357290 23504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:55.358453 23504 hybrid_clock.cc:648] HybridClock initialized: now 1786079335358418 us; error 45 us; skew 500 ppm
Aug 07 05:08:55 dist-test-slave-1x4g krb5kdc[23491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079335, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:55.361300 23504 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 05:08:55.362248 23504 webserver.cc:460] Webserver started at http://127.0.0.1:40687/ using document root <none> and password file <none>
I20260807 05:08:55.362852 23504 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:55.362913 23504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:55.363132 23504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:55.365356 23504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "3a1cb594a1a54976be9ea27183ee5d62"
format_stamp: "Formatted at 2026-08-07 05:08:55 on dist-test-slave-1x4g"
server_key: "506c755e03b2e7089cb302fce054b46b"
server_key_iv: "db6c610dafe65893f03a7fdcd0f0d9cb"
server_key_version: "encryptionkey@0"
I20260807 05:08:55.365957 23504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "3a1cb594a1a54976be9ea27183ee5d62"
format_stamp: "Formatted at 2026-08-07 05:08:55 on dist-test-slave-1x4g"
server_key: "506c755e03b2e7089cb302fce054b46b"
server_key_iv: "db6c610dafe65893f03a7fdcd0f0d9cb"
server_key_version: "encryptionkey@0"
I20260807 05:08:55.369894 23504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 05:08:55.372340 23520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:55.373656 23504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:55.373795 23504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "3a1cb594a1a54976be9ea27183ee5d62"
format_stamp: "Formatted at 2026-08-07 05:08:55 on dist-test-slave-1x4g"
server_key: "506c755e03b2e7089cb302fce054b46b"
server_key_iv: "db6c610dafe65893f03a7fdcd0f0d9cb"
server_key_version: "encryptionkey@0"
I20260807 05:08:55.373910 23504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:55.378613 23504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:55.381479 23504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:55.381664 23504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:55.388752 23504 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43701
I20260807 05:08:55.388828 23575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43701 every 8 connection(s)
I20260807 05:08:55.389786 23504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:55.392521 23576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:55.397992 23576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62: Bootstrap starting.
I20260807 05:08:55.400067 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23504
I20260807 05:08:55.400206 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:55.400485 23576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:55.400506 1095 external_mini_cluster.cc:1428] Setting key 7a465f742998cd22b69928d6ca7e9e41
I20260807 05:08:55.401343 23576 log.cc:824] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:55.403321 23576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62: No bootstrap required, opened a new log
I20260807 05:08:55.405501 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:55.401649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:37789 (local address 127.0.0.1:43701)
0807 05:08:55.402130 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:55.402151 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:55.402197 (+ 46us) server_negotiation.cc:407] Connection header received
0807 05:08:55.402870 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:55.402887 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:55.403153 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:55.403495 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:55.404263 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:55.404268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:55.404279 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:55.404289 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:55.404341 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:55.404361 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:55.404477 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:55.404599 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 05:08:55.404756 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":149,"threads_started":1}
I20260807 05:08:55.407647 23576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cb594a1a54976be9ea27183ee5d62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260807 05:08:55.407889 23576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:55.407959 23576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1cb594a1a54976be9ea27183ee5d62, State: Initialized, Role: FOLLOWER
I20260807 05:08:55.408543 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:55.408520 23576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cb594a1a54976be9ea27183ee5d62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260807 05:08:55.408672 23576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:55.408767 23576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:55.408895 23576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:55.409813 23576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cb594a1a54976be9ea27183ee5d62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260807 05:08:55.410216 23576 leader_election.cc:302] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a1cb594a1a54976be9ea27183ee5d62; no voters:
I20260807 05:08:55.410558 23576 leader_election.cc:288] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:55.410666 23586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:55.410979 23586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1cb594a1a54976be9ea27183ee5d62, State: Running, Role: LEADER
I20260807 05:08:55.411319 23586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cb594a1a54976be9ea27183ee5d62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260807 05:08:55.411918 23576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:55.412829 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:55.410724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:40147 (local address 127.0.0.1:43701)
0807 05:08:55.410830 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:55.410834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:55.411095 (+ 261us) server_negotiation.cc:407] Connection header received
0807 05:08:55.411455 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:55.411458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:55.411494 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:55.411575 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:55.412276 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:55.412280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:55.412282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:55.412294 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:55.412323 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:55.412339 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:55.412445 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:55.412597 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 05:08:55.412645 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 05:08:55.413031 23589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a1cb594a1a54976be9ea27183ee5d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cb594a1a54976be9ea27183ee5d62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } } }
I20260807 05:08:55.413193 23589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:55.413335 23588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a1cb594a1a54976be9ea27183ee5d62. Latest consensus state: current_term: 1 leader_uuid: "3a1cb594a1a54976be9ea27183ee5d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1cb594a1a54976be9ea27183ee5d62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } } }
I20260807 05:08:55.413497 23588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62 [sys.catalog]: This master's current role is: LEADER
W20260807 05:08:55.416867 23602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:55.416952 23602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:55.417022 23596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:55.417702 23596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:55.422962 23596 catalog_manager.cc:1347] Generated new cluster ID: 1ba72e30c75244db9409090d2c2797bd
I20260807 05:08:55.423022 23596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:55.439155 23596 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:55.439503 23596 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:55.464216 23596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a1cb594a1a54976be9ea27183ee5d62: Generated new TSK 0
I20260807 05:08:55.465026 23596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:55.479007 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23504
2026-08-07T05:08:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:55 dist-test-slave-1x4g krb5kdc[23608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:55 dist-test-slave-1x4g krb5kdc[23608](info): set up 2 sockets
Aug 07 05:08:55 dist-test-slave-1x4g krb5kdc[23608](info): commencing operation
krb5kdc: starting...
W20260807 05:08:56.529920 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:56 dist-test-slave-1x4g krb5kdc[23608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:08:56.651865 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:42497
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:42497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:56.761399 23623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:56.761621 23623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:56.761659 23623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:56.765506 23623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:56.765551 23623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:56.765569 23623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:56.765585 23623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:56.765641 23623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:56.770651 23623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:42497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:42497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:56.771580 23623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:56.772523 23623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:08:56.781232 23623 server_base.cc:1034] running on GCE node
W20260807 05:08:56.781196 23631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:56.781189 23628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:56.781492 23629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:56.782199 23623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:56.783495 23623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:56.784664 23623 hybrid_clock.cc:648] HybridClock initialized: now 1786079336784643 us; error 31 us; skew 500 ppm
Aug 07 05:08:56 dist-test-slave-1x4g krb5kdc[23608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079336, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:56.792320 23623 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:08:56.793292 23623 webserver.cc:460] Webserver started at http://127.1.17.254:43225/ using document root <none> and password file <none>
I20260807 05:08:56.793824 23623 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:56.793896 23623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:56.794071 23623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:56.796108 23623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "0a487a5073934c98a026fa324eb09cee"
format_stamp: "Formatted at 2026-08-07 05:08:56 on dist-test-slave-1x4g"
server_key: "881cf9b556a83ba6bf92ad6f5bee32f1"
server_key_iv: "ebfc24395c3d8737cbd8e2bcc77be192"
server_key_version: "encryptionkey@0"
I20260807 05:08:56.796671 23623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "0a487a5073934c98a026fa324eb09cee"
format_stamp: "Formatted at 2026-08-07 05:08:56 on dist-test-slave-1x4g"
server_key: "881cf9b556a83ba6bf92ad6f5bee32f1"
server_key_iv: "ebfc24395c3d8737cbd8e2bcc77be192"
server_key_version: "encryptionkey@0"
I20260807 05:08:56.800655 23623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 05:08:56.803141 23638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:56.804381 23623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:56.804538 23623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "0a487a5073934c98a026fa324eb09cee"
format_stamp: "Formatted at 2026-08-07 05:08:56 on dist-test-slave-1x4g"
server_key: "881cf9b556a83ba6bf92ad6f5bee32f1"
server_key_iv: "ebfc24395c3d8737cbd8e2bcc77be192"
server_key_version: "encryptionkey@0"
I20260807 05:08:56.804651 23623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:56.826277 23623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:56.829679 23623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:56.829928 23623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:56.839687 23623 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:42497
I20260807 05:08:56.839699 23691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:42497 every 8 connection(s)
I20260807 05:08:56.840674 23623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:56.843555 23692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:56.847927 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23623
I20260807 05:08:56.848120 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:56.848415 1095 external_mini_cluster.cc:1428] Setting key a236d39f7c82118c95b8874571c418db
I20260807 05:08:56.849714 23692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee: Bootstrap starting.
I20260807 05:08:56.852496 23692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:56.853468 23692 log.cc:824] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:56.855566 23692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee: No bootstrap required, opened a new log
Aug 07 05:08:56 dist-test-slave-1x4g krb5kdc[23608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:08:56.859555 23692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a487a5073934c98a026fa324eb09cee" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42497 } }
I20260807 05:08:56.859896 23692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:56.859972 23692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a487a5073934c98a026fa324eb09cee, State: Initialized, Role: FOLLOWER
I20260807 05:08:56.860482 23692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a487a5073934c98a026fa324eb09cee" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42497 } }
I20260807 05:08:56.860666 23692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:56.860726 23692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:56.860852 23692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:56.861968 23692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a487a5073934c98a026fa324eb09cee" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42497 } }
I20260807 05:08:56.862347 23692 leader_election.cc:302] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a487a5073934c98a026fa324eb09cee; no voters:
I20260807 05:08:56.862694 23692 leader_election.cc:288] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:56.862797 23697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:56.863044 23697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [term 1 LEADER]: Becoming Leader. State: Replica: 0a487a5073934c98a026fa324eb09cee, State: Running, Role: LEADER
I20260807 05:08:56.863435 23697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a487a5073934c98a026fa324eb09cee" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42497 } }
I20260807 05:08:56.864028 23692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:56.864545 23695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:56.849945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37149 (local address 127.1.17.254:42497)
0807 05:08:56.850374 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:56.850393 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:56.850434 (+ 41us) server_negotiation.cc:407] Connection header received
0807 05:08:56.851270 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:56.851293 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:56.851733 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:56.852130 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:56.852919 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:56.854268 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:56.855278 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:56.855977 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:56.856128 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:56.858786 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:56.858792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:56.858808 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:56.858838 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:56.860876 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:56.861496 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:56.861501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:56.861505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:56.861616 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:56.861997 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:56.862000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:56.862001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:56.862502 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:56.862734 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:56.863371 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 05:08:56.863590 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":120,"threads_started":1}
I20260807 05:08:56.864565 23699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a487a5073934c98a026fa324eb09cee. Latest consensus state: current_term: 1 leader_uuid: "0a487a5073934c98a026fa324eb09cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a487a5073934c98a026fa324eb09cee" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42497 } } }
I20260807 05:08:56.864737 23699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:56.865128 23698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a487a5073934c98a026fa324eb09cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a487a5073934c98a026fa324eb09cee" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 42497 } } }
I20260807 05:08:56.865247 23698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:56.865197 23702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:56.868526 23702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:56.870069 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:56.876075 23702 catalog_manager.cc:1347] Generated new cluster ID: 7e0f93045e864941a9e277531001dc81
I20260807 05:08:56.876137 23702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:56.880942 23695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:56.872312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47807 (local address 127.1.17.254:42497)
0807 05:08:56.872402 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:56.872407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:56.872734 (+ 327us) server_negotiation.cc:407] Connection header received
0807 05:08:56.872949 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:56.872952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:56.872996 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:56.873047 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:56.873972 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:56.874499 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:56.875589 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:56.875914 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:56.875992 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:56.877622 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:56.877625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:56.877628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:08:56.877652 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:56.879079 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:56.879556 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:56.879559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:56.879561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:56.879657 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:08:56.879998 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:08:56.880001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:08:56.880003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:08:56.880170 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:56.880338 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:56.880616 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 05:08:56.880708 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 05:08:56.888751 23702 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:56.889900 23702 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:56.899181 23702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a487a5073934c98a026fa324eb09cee: Generated new TSK 0
I20260807 05:08:56.899946 23702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:56.909176 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23623
W20260807 05:08:56.916249 23606 connection.cc:535] client connection to 127.1.17.254:42497 recv error: Network error: failed to read from TLS socket (remote: 127.1.17.254:42497): Connection reset by peer (error 104)
2026-08-07T05:08:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:56 dist-test-slave-1x4g krb5kdc[23723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:56 dist-test-slave-1x4g krb5kdc[23723](info): set up 2 sockets
Aug 07 05:08:56 dist-test-slave-1x4g krb5kdc[23723](info): commencing operation
krb5kdc: starting...
W20260807 05:08:57.945879 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:58 dist-test-slave-1x4g krb5kdc[23723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:08:58.069931 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38661
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:38661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:58.179734 23737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:58.180011 23737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:58.180080 23737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:58.183867 23737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:58.183966 23737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:58.184006 23737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:58.184042 23737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:58.184078 23737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:58.189090 23737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:38661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:58.190063 23737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:58.191022 23737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:58.199263 23743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:58.199254 23742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:58.199460 23737 server_base.cc:1034] running on GCE node
W20260807 05:08:58.199347 23745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:58.200169 23737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:58.201306 23737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:58.202492 23737 hybrid_clock.cc:648] HybridClock initialized: now 1786079338202466 us; error 41 us; skew 500 ppm
Aug 07 05:08:58 dist-test-slave-1x4g krb5kdc[23723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079338, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:58.205175 23737 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:08:58.205991 23737 webserver.cc:460] Webserver started at http://127.1.17.254:46569/ using document root <none> and password file <none>
I20260807 05:08:58.206519 23737 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:58.206578 23737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:58.206815 23737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:58.208751 23737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "c41f9e69ab4a4404bc11e0b064572d6c"
format_stamp: "Formatted at 2026-08-07 05:08:58 on dist-test-slave-1x4g"
server_key: "2a72d655f1e3f4f08b7cc06e532e343a"
server_key_iv: "75415e2f7d4d0f97f27a59a90f9a5446"
server_key_version: "encryptionkey@0"
I20260807 05:08:58.209362 23737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "c41f9e69ab4a4404bc11e0b064572d6c"
format_stamp: "Formatted at 2026-08-07 05:08:58 on dist-test-slave-1x4g"
server_key: "2a72d655f1e3f4f08b7cc06e532e343a"
server_key_iv: "75415e2f7d4d0f97f27a59a90f9a5446"
server_key_version: "encryptionkey@0"
I20260807 05:08:58.212917 23737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:58.215245 23752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:58.216409 23737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:08:58.216542 23737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "c41f9e69ab4a4404bc11e0b064572d6c"
format_stamp: "Formatted at 2026-08-07 05:08:58 on dist-test-slave-1x4g"
server_key: "2a72d655f1e3f4f08b7cc06e532e343a"
server_key_iv: "75415e2f7d4d0f97f27a59a90f9a5446"
server_key_version: "encryptionkey@0"
I20260807 05:08:58.216657 23737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:58.246119 23737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:58.249079 23737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:58.249282 23737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:58.256351 23737 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:38661
I20260807 05:08:58.256378 23808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:38661 every 8 connection(s)
I20260807 05:08:58.257282 23737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:58.259972 23809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:58.265405 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c: Bootstrap starting.
I20260807 05:08:58.265836 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23737
I20260807 05:08:58.266007 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:58.266299 1095 external_mini_cluster.cc:1428] Setting key 0058fc7fdbc9dedaa156ea4479041e10
I20260807 05:08:58.268167 23809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:58.268959 23809 log.cc:824] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:58.270938 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c: No bootstrap required, opened a new log
I20260807 05:08:58.275347 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:58.267619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49631 (local address 127.1.17.254:38661)
0807 05:08:58.268092 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:58.268107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:58.268155 (+ 48us) server_negotiation.cc:407] Connection header received
0807 05:08:58.268863 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:58.268882 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:58.269242 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:58.269620 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:58.269986 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:58.270884 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:58.272070 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:58.272594 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:58.272743 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:58.273645 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:58.273650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:58.273662 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:58.274065 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:58.274118 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:58.274140 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:58.274243 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:58.274456 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 05:08:58.274608 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":128,"threads_started":1}
I20260807 05:08:58.275280 23809 raft_consensus.cc:348] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38661 } }
I20260807 05:08:58.275534 23809 raft_consensus.cc:374] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:58.275616 23809 raft_consensus.cc:729] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41f9e69ab4a4404bc11e0b064572d6c, State: Initialized, Role: FOLLOWER
I20260807 05:08:58.276074 23809 consensus_queue.cc:260] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38661 } }
I20260807 05:08:58.276229 23809 raft_consensus.cc:388] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:58.276302 23809 raft_consensus.cc:482] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:58.276402 23809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:58.277451 23809 raft_consensus.cc:504] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38661 } }
I20260807 05:08:58.277842 23809 leader_election.cc:302] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c41f9e69ab4a4404bc11e0b064572d6c; no voters:
I20260807 05:08:58.278213 23809 leader_election.cc:288] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:58.278278 23814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:58.278443 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:58.278620 23814 raft_consensus.cc:686] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [term 1 LEADER]: Becoming Leader. State: Replica: c41f9e69ab4a4404bc11e0b064572d6c, State: Running, Role: LEADER
I20260807 05:08:58.278963 23814 consensus_queue.cc:237] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38661 } }
I20260807 05:08:58.279657 23809 sys_catalog.cc:564] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:58.280169 23814 sys_catalog.cc:455] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c41f9e69ab4a4404bc11e0b064572d6c. Latest consensus state: current_term: 1 leader_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38661 } } }
I20260807 05:08:58.280354 23814 sys_catalog.cc:458] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:58.280596 23815 sys_catalog.cc:455] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c41f9e69ab4a4404bc11e0b064572d6c" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38661 } } }
I20260807 05:08:58.280716 23815 sys_catalog.cc:458] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:58.280812 23824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:58.284031 23824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:58.286271 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:58.281365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38379 (local address 127.1.17.254:38661)
0807 05:08:58.281461 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:58.281466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:58.281707 (+ 241us) server_negotiation.cc:407] Connection header received
0807 05:08:58.281882 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:58.281886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:58.281927 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:58.281978 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:58.282355 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:58.283096 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:58.284304 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:08:58.284609 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:08:58.284667 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:08:58.285726 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:58.285729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:58.285732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:58.285743 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:58.285777 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:58.285789 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:58.285901 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:58.286075 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 05:08:58.286110 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 05:08:58.289897 23824 catalog_manager.cc:1347] Generated new cluster ID: 6106b042e56349f0a17538cc7b3a12f7
I20260807 05:08:58.289973 23824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:58.303893 23824 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:58.305052 23824 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:58.314317 23824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c41f9e69ab4a4404bc11e0b064572d6c: Generated new TSK 0
I20260807 05:08:58.315084 23824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:58.319568 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23737
W20260807 05:08:58.326403 23721 connection.cc:535] client connection to 127.1.17.254:38661 recv error: Network error: failed to read from TLS socket (remote: 127.1.17.254:38661): Connection reset by peer (error 104)
2026-08-07T05:08:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:58 dist-test-slave-1x4g krb5kdc[23840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:58 dist-test-slave-1x4g krb5kdc[23840](info): set up 2 sockets
Aug 07 05:08:58 dist-test-slave-1x4g krb5kdc[23840](info): commencing operation
krb5kdc: starting...
W20260807 05:08:59.361922 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:08:59 dist-test-slave-1x4g krb5kdc[23840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:08:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:08:59.488072 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43749
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:43749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:08:59.596592 23853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:08:59.596952 23853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:08:59.597045 23853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:08:59.600850 23853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:08:59.600927 23853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:08:59.600963 23853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:08:59.600997 23853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:08:59.601042 23853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:08:59.606050 23853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:43749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:43749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:08:59.606959 23853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:08:59.607903 23853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:08:59.616094 23858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:08:59.616128 23859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:59.616217 23853 server_base.cc:1034] running on GCE node
W20260807 05:08:59.616099 23861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:08:59.616948 23853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:08:59.617985 23853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:08:59.619145 23853 hybrid_clock.cc:648] HybridClock initialized: now 1786079339619127 us; error 29 us; skew 500 ppm
Aug 07 05:08:59 dist-test-slave-1x4g krb5kdc[23840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079339, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:08:59.621970 23853 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:08:59.622728 23853 webserver.cc:460] Webserver started at http://127.1.17.254:43009/ using document root <none> and password file <none>
I20260807 05:08:59.623236 23853 fs_manager.cc:359] Metadata directory not provided
I20260807 05:08:59.623292 23853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:08:59.623456 23853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:08:59.625501 23853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "84b7eabe6cb741439de15ccc850b690d"
format_stamp: "Formatted at 2026-08-07 05:08:59 on dist-test-slave-1x4g"
server_key: "c82bab25bcd36b40d750d10c2f64ca12"
server_key_iv: "775fa94e19b0aa0a77ba4a189dac4fb5"
server_key_version: "encryptionkey@0"
I20260807 05:08:59.625998 23853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "84b7eabe6cb741439de15ccc850b690d"
format_stamp: "Formatted at 2026-08-07 05:08:59 on dist-test-slave-1x4g"
server_key: "c82bab25bcd36b40d750d10c2f64ca12"
server_key_iv: "775fa94e19b0aa0a77ba4a189dac4fb5"
server_key_version: "encryptionkey@0"
I20260807 05:08:59.629521 23853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 05:08:59.631767 23868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:08:59.633082 23853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:08:59.633208 23853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "84b7eabe6cb741439de15ccc850b690d"
format_stamp: "Formatted at 2026-08-07 05:08:59 on dist-test-slave-1x4g"
server_key: "c82bab25bcd36b40d750d10c2f64ca12"
server_key_iv: "775fa94e19b0aa0a77ba4a189dac4fb5"
server_key_version: "encryptionkey@0"
I20260807 05:08:59.633318 23853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:08:59.637663 23853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:08:59.640339 23853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:08:59.640529 23853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:08:59.647624 23853 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:43749
I20260807 05:08:59.647651 23922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:43749 every 8 connection(s)
I20260807 05:08:59.648499 23853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:08:59.651047 23923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:08:59.653867 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23853
I20260807 05:08:59.654036 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:08:59.654345 1095 external_mini_cluster.cc:1428] Setting key e201810f96f9416afd7afb26054ee038
I20260807 05:08:59.657027 23923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d: Bootstrap starting.
I20260807 05:08:59.659293 23923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d: Neither blocks nor log segments found. Creating new log.
I20260807 05:08:59.659459 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:59.655734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41993 (local address 127.1.17.254:43749)
0807 05:08:59.656148 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:59.656162 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:59.656196 (+ 34us) server_negotiation.cc:407] Connection header received
0807 05:08:59.656901 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:59.656920 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:59.657228 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:59.657553 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:59.658305 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:59.658309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:59.658321 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:59.658331 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:59.658387 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:59.658406 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:59.658548 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:59.658619 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 05:08:59.658763 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":126,"threads_started":1}
I20260807 05:08:59.660049 23923 log.cc:824] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d: Log is configured to *not* fsync() on all Append() calls
I20260807 05:08:59.661963 23923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d: No bootstrap required, opened a new log
I20260807 05:08:59.662204 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:08:59.666096 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:08:59.664394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44947 (local address 127.1.17.254:43749)
0807 05:08:59.664522 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:08:59.664526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:08:59.664740 (+ 214us) server_negotiation.cc:407] Connection header received
0807 05:08:59.664880 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:08:59.664882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:08:59.664929 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:08:59.664999 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:08:59.665683 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:08:59.665686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:08:59.665688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:08:59.665698 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:08:59.665720 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:08:59.665729 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:08:59.665798 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 05:08:59.665906 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 05:08:59.665953 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 05:08:59.666694 23923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b7eabe6cb741439de15ccc850b690d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43749 } }
I20260807 05:08:59.666956 23923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:08:59.667019 23923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b7eabe6cb741439de15ccc850b690d, State: Initialized, Role: FOLLOWER
I20260807 05:08:59.667454 23923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b7eabe6cb741439de15ccc850b690d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43749 } }
I20260807 05:08:59.667598 23923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:08:59.667662 23923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:08:59.667755 23923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:08:59.668571 23923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b7eabe6cb741439de15ccc850b690d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43749 } }
I20260807 05:08:59.668936 23923 leader_election.cc:302] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84b7eabe6cb741439de15ccc850b690d; no voters:
I20260807 05:08:59.669266 23923 leader_election.cc:288] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:08:59.669373 23934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:08:59.669629 23934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [term 1 LEADER]: Becoming Leader. State: Replica: 84b7eabe6cb741439de15ccc850b690d, State: Running, Role: LEADER
I20260807 05:08:59.669973 23934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b7eabe6cb741439de15ccc850b690d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43749 } }
I20260807 05:08:59.670392 23923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:08:59.671496 23936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84b7eabe6cb741439de15ccc850b690d. Latest consensus state: current_term: 1 leader_uuid: "84b7eabe6cb741439de15ccc850b690d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b7eabe6cb741439de15ccc850b690d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43749 } } }
I20260807 05:08:59.672246 23936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [sys.catalog]: This master's current role is: LEADER
I20260807 05:08:59.671985 23935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84b7eabe6cb741439de15ccc850b690d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b7eabe6cb741439de15ccc850b690d" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 43749 } } }
I20260807 05:08:59.672423 23935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d [sys.catalog]: This master's current role is: LEADER
W20260807 05:08:59.674860 23949 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:08:59.674952 23949 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:08:59.675033 23950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:08:59.675729 23950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:08:59.680625 23950 catalog_manager.cc:1347] Generated new cluster ID: 10f1d8e8c1e9452c8f2f4249ee29a50e
I20260807 05:08:59.680684 23950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:08:59.695824 23950 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:08:59.696147 23950 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:08:59.703536 23950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84b7eabe6cb741439de15ccc850b690d: Generated new TSK 0
I20260807 05:08:59.704154 23950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:08:59.713649 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23853
2026-08-07T05:08:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:08:59 dist-test-slave-1x4g krb5kdc[23957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:08:59 dist-test-slave-1x4g krb5kdc[23957](info): set up 2 sockets
Aug 07 05:08:59 dist-test-slave-1x4g krb5kdc[23957](info): commencing operation
krb5kdc: starting...
W20260807 05:09:00.761161 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:09:00 dist-test-slave-1x4g krb5kdc[23957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:09:00.895349 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39721
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:39721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:01.010687 23971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:01.010926 23971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:01.010972 23971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:01.014899 23971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:01.014957 23971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:01.014976 23971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:01.015018 23971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:01.015050 23971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:01.020161 23971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:39721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:01.021214 23971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:01.022181 23971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:09:01.030813 23971 server_base.cc:1034] running on GCE node
W20260807 05:09:01.030777 23976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:01.030802 23979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:01.030773 23977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:01.031626 23971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:01.032734 23971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:01.033972 23971 hybrid_clock.cc:648] HybridClock initialized: now 1786079341033951 us; error 31 us; skew 500 ppm
Aug 07 05:09:01 dist-test-slave-1x4g krb5kdc[23957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079341, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:01.036998 23971 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:09:01.037894 23971 webserver.cc:460] Webserver started at http://127.1.17.254:36671/ using document root <none> and password file <none>
I20260807 05:09:01.038450 23971 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:01.038511 23971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:01.038758 23971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:01.040912 23971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "d17e1f5516f74297b4e83b52da799485"
format_stamp: "Formatted at 2026-08-07 05:09:01 on dist-test-slave-1x4g"
server_key: "9047751a9b34567e7f1fa1b2dd1dce74"
server_key_iv: "0b027f108a90059816e6fa20d826be0a"
server_key_version: "encryptionkey@0"
I20260807 05:09:01.041553 23971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "d17e1f5516f74297b4e83b52da799485"
format_stamp: "Formatted at 2026-08-07 05:09:01 on dist-test-slave-1x4g"
server_key: "9047751a9b34567e7f1fa1b2dd1dce74"
server_key_iv: "0b027f108a90059816e6fa20d826be0a"
server_key_version: "encryptionkey@0"
I20260807 05:09:01.045301 23971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 05:09:01.047753 23986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:01.049069 23971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:01.049203 23971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "d17e1f5516f74297b4e83b52da799485"
format_stamp: "Formatted at 2026-08-07 05:09:01 on dist-test-slave-1x4g"
server_key: "9047751a9b34567e7f1fa1b2dd1dce74"
server_key_iv: "0b027f108a90059816e6fa20d826be0a"
server_key_version: "encryptionkey@0"
I20260807 05:09:01.049312 23971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:01.065817 23971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:01.068645 23971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:01.068866 23971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:01.075989 23971 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:39721
I20260807 05:09:01.076010 24041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:39721 every 8 connection(s)
I20260807 05:09:01.076967 23971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:01.079777 24042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:01.082108 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 23971
I20260807 05:09:01.082261 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:09:01.082540 1095 external_mini_cluster.cc:1428] Setting key ba6d5f30b11e7c5455358b98f737e45e
I20260807 05:09:01.085755 24042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485: Bootstrap starting.
I20260807 05:09:01.088270 24042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:01.089133 24042 log.cc:824] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:01.091434 24042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485: No bootstrap required, opened a new log
Aug 07 05:09:01 dist-test-slave-1x4g krb5kdc[23957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:09:01.095737 24042 raft_consensus.cc:348] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e1f5516f74297b4e83b52da799485" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39721 } }
I20260807 05:09:01.096017 24042 raft_consensus.cc:374] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:01.096079 24042 raft_consensus.cc:729] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d17e1f5516f74297b4e83b52da799485, State: Initialized, Role: FOLLOWER
I20260807 05:09:01.096581 24042 consensus_queue.cc:260] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e1f5516f74297b4e83b52da799485" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39721 } }
I20260807 05:09:01.096760 24042 raft_consensus.cc:388] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:01.096880 24042 raft_consensus.cc:482] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:01.096992 24042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:01.098047 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:01.083859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43483 (local address 127.1.17.254:39721)
0807 05:09:01.084264 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:01.084279 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:01.084315 (+ 36us) server_negotiation.cc:407] Connection header received
0807 05:09:01.085015 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:01.085034 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:01.085367 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:01.085638 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:01.086574 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:01.087595 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:01.088759 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:01.089343 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:01.089499 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:01.092250 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:01.092256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:01.092271 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:01.092301 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:01.094831 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:01.095279 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:01.095285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:01.095290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:01.095426 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:01.095705 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:01.095707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:01.095709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:01.096222 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:01.096412 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:01.096734 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 05:09:01.097065 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":127,"threads_started":1}
I20260807 05:09:01.098066 24042 raft_consensus.cc:504] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e1f5516f74297b4e83b52da799485" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39721 } }
I20260807 05:09:01.098397 24042 leader_election.cc:302] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d17e1f5516f74297b4e83b52da799485; no voters:
I20260807 05:09:01.098747 24042 leader_election.cc:288] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:01.098872 24048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:01.099108 24048 raft_consensus.cc:686] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [term 1 LEADER]: Becoming Leader. State: Replica: d17e1f5516f74297b4e83b52da799485, State: Running, Role: LEADER
I20260807 05:09:01.099475 24048 consensus_queue.cc:237] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e1f5516f74297b4e83b52da799485" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39721 } }
I20260807 05:09:01.099963 24042 sys_catalog.cc:564] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:01.101300 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:01.101433 24050 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d17e1f5516f74297b4e83b52da799485. Latest consensus state: current_term: 1 leader_uuid: "d17e1f5516f74297b4e83b52da799485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e1f5516f74297b4e83b52da799485" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39721 } } }
I20260807 05:09:01.101540 24050 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:01.101867 24049 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d17e1f5516f74297b4e83b52da799485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e1f5516f74297b4e83b52da799485" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39721 } } }
I20260807 05:09:01.101960 24049 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485 [sys.catalog]: This master's current role is: LEADER
W20260807 05:09:01.105279 24063 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:01.105391 24063 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:01.105468 24060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:01.106098 24060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:01.111227 24060 catalog_manager.cc:1347] Generated new cluster ID: 874ff0fc6ea04fa086fb532ce32012c0
I20260807 05:09:01.111316 24060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:01.116112 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:01.105290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54993 (local address 127.1.17.254:39721)
0807 05:09:01.105401 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:01.105407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:01.107747 (+ 2340us) server_negotiation.cc:407] Connection header received
0807 05:09:01.107894 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:01.107897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:01.107944 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:01.108033 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:01.108639 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:01.109615 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:01.110739 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:01.111049 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:01.111159 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:01.112681 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:01.112685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:01.112687 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:01.112708 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:01.114462 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:01.114952 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:01.114955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:01.114957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:01.115032 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:01.115323 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:01.115326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:01.115328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:01.115506 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:01.115613 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:01.115853 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 05:09:01.115955 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 05:09:01.130493 24060 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:01.131279 24060 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:01.143648 24060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d17e1f5516f74297b4e83b52da799485: Generated new TSK 0
I20260807 05:09:01.144292 24060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:01.152720 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 23971
2026-08-07T05:09:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:01 dist-test-slave-1x4g krb5kdc[24074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:01 dist-test-slave-1x4g krb5kdc[24074](info): set up 2 sockets
Aug 07 05:09:01 dist-test-slave-1x4g krb5kdc[24074](info): commencing operation
krb5kdc: starting...
W20260807 05:09:02.201622 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:09:02 dist-test-slave-1x4g krb5kdc[24074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:09:02.329170 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38319
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:33667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:38319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:02.440914 24087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:02.441156 24087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:02.441223 24087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:02.445104 24087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:02.445148 24087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:02.445165 24087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:02.445223 24087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:02.445254 24087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:02.450093 24087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:33667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:38319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:38319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:02.451023 24087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:02.451984 24087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:02.460412 24093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:02.460430 24097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:02.460399 24094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:02.460695 24087 server_base.cc:1034] running on GCE node
I20260807 05:09:02.461453 24087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:02.462522 24087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:02.463726 24087 hybrid_clock.cc:648] HybridClock initialized: now 1786079342463712 us; error 29 us; skew 500 ppm
Aug 07 05:09:02 dist-test-slave-1x4g krb5kdc[24074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079342, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:02.473134 24087 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:09:02.474141 24087 webserver.cc:460] Webserver started at http://127.1.17.254:34823/ using document root <none> and password file <none>
I20260807 05:09:02.474709 24087 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:02.474798 24087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:02.475050 24087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:02.477169 24087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "da5edeef14a24a6ea8a24c8ccdd39386"
format_stamp: "Formatted at 2026-08-07 05:09:02 on dist-test-slave-1x4g"
server_key: "d1313a455df11b1d28abb45f47c85e5d"
server_key_iv: "711852771403094f71494a77239b370c"
server_key_version: "encryptionkey@0"
I20260807 05:09:02.477738 24087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "da5edeef14a24a6ea8a24c8ccdd39386"
format_stamp: "Formatted at 2026-08-07 05:09:02 on dist-test-slave-1x4g"
server_key: "d1313a455df11b1d28abb45f47c85e5d"
server_key_iv: "711852771403094f71494a77239b370c"
server_key_version: "encryptionkey@0"
I20260807 05:09:02.481468 24087 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 05:09:02.483882 24106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:02.485108 24087 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:02.485255 24087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "da5edeef14a24a6ea8a24c8ccdd39386"
format_stamp: "Formatted at 2026-08-07 05:09:02 on dist-test-slave-1x4g"
server_key: "d1313a455df11b1d28abb45f47c85e5d"
server_key_iv: "711852771403094f71494a77239b370c"
server_key_version: "encryptionkey@0"
I20260807 05:09:02.485371 24087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:02.502924 24087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:02.505798 24087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:02.506007 24087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:02.513381 24087 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:38319
I20260807 05:09:02.513402 24162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:38319 every 8 connection(s)
I20260807 05:09:02.514357 24087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:02.515610 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24087
I20260807 05:09:02.515743 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:09:02.516083 1095 external_mini_cluster.cc:1428] Setting key fb1b106f77db313702819e756de27477
I20260807 05:09:02.517547 24163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:02.525049 24163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386: Bootstrap starting.
I20260807 05:09:02.526961 24166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:02.517635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43985 (local address 127.1.17.254:38319)
0807 05:09:02.518112 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:02.518131 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:02.518173 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:09:02.519039 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:02.519065 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:02.519493 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:02.519845 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:02.520488 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:02.522163 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:02.523121 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:02.523862 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:02.524055 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:02.524876 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:02.524884 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:02.524899 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:02.525427 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:02.525504 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:02.525537 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:02.525649 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:02.525755 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 05:09:02.525971 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
I20260807 05:09:02.527776 24163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:02.528674 24163 log.cc:824] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:02.530107 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:02.531126 24163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386: No bootstrap required, opened a new log
I20260807 05:09:02.535622 24163 raft_consensus.cc:348] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38319 } }
I20260807 05:09:02.535892 24163 raft_consensus.cc:374] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:02.535957 24163 raft_consensus.cc:729] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da5edeef14a24a6ea8a24c8ccdd39386, State: Initialized, Role: FOLLOWER
I20260807 05:09:02.536433 24163 consensus_queue.cc:260] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38319 } }
I20260807 05:09:02.536542 24166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:02.532454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39847 (local address 127.1.17.254:38319)
0807 05:09:02.532574 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:02.532579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:02.532720 (+ 141us) server_negotiation.cc:407] Connection header received
0807 05:09:02.532849 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:02.532852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:02.532904 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:02.532992 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:02.533229 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:02.534079 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:02.534846 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:02.535200 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:02.535313 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:02.536126 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:02.536130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:02.536133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:02.536145 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:02.536175 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:02.536187 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:02.536260 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:02.536333 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 05:09:02.536382 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:02.536635 24163 raft_consensus.cc:388] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:02.536700 24163 raft_consensus.cc:482] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:02.536849 24163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:02.537827 24163 raft_consensus.cc:504] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38319 } }
I20260807 05:09:02.538178 24163 leader_election.cc:302] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da5edeef14a24a6ea8a24c8ccdd39386; no voters:
I20260807 05:09:02.538484 24163 leader_election.cc:288] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:02.538630 24174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:02.538872 24174 raft_consensus.cc:686] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [term 1 LEADER]: Becoming Leader. State: Replica: da5edeef14a24a6ea8a24c8ccdd39386, State: Running, Role: LEADER
I20260807 05:09:02.539168 24174 consensus_queue.cc:237] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38319 } }
I20260807 05:09:02.539839 24163 sys_catalog.cc:564] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:02.540966 24175 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38319 } } }
I20260807 05:09:02.541088 24175 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:02.541353 24176 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da5edeef14a24a6ea8a24c8ccdd39386. Latest consensus state: current_term: 1 leader_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5edeef14a24a6ea8a24c8ccdd39386" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 38319 } } }
I20260807 05:09:02.541433 24176 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:02.541494 24184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:02.545024 24184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:02.550978 24184 catalog_manager.cc:1347] Generated new cluster ID: bc973c9e43a64a7d9209fb7ec06a673b
I20260807 05:09:02.551048 24184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:02.561949 24184 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:02.562775 24184 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:02.566840 24184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da5edeef14a24a6ea8a24c8ccdd39386: Generated new TSK 0
I20260807 05:09:02.567440 24184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:02.571720 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24087
2026-08-07T05:09:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:02 dist-test-slave-1x4g krb5kdc[24196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:02 dist-test-slave-1x4g krb5kdc[24196](info): set up 2 sockets
Aug 07 05:09:02 dist-test-slave-1x4g krb5kdc[24196](info): commencing operation
krb5kdc: starting...
W20260807 05:09:03.633874 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:09:03 dist-test-slave-1x4g krb5kdc[24196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:09:03.759548 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:37809
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:32839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:37809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:03.870865 24209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:03.871102 24209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:03.871140 24209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:03.874919 24209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:03.874974 24209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:03.874992 24209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:03.875007 24209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:03.875022 24209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:03.879913 24209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:32839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:37809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:37809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:03.880893 24209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:03.881819 24209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:03.890084 24217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:03.890105 24215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:03.890136 24214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:03.890262 24209 server_base.cc:1034] running on GCE node
I20260807 05:09:03.890894 24209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:03.891957 24209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:03.893115 24209 hybrid_clock.cc:648] HybridClock initialized: now 1786079343893102 us; error 34 us; skew 500 ppm
Aug 07 05:09:03 dist-test-slave-1x4g krb5kdc[24196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079343, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:03.895900 24209 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:09:03.896836 24209 webserver.cc:460] Webserver started at http://127.1.17.254:33471/ using document root <none> and password file <none>
I20260807 05:09:03.897380 24209 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:03.897469 24209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:03.897697 24209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:03.899658 24209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "562a6035ea9444f181e9f26a61a444fc"
format_stamp: "Formatted at 2026-08-07 05:09:03 on dist-test-slave-1x4g"
server_key: "a26ba904d7592cc2ddb0cc856367723f"
server_key_iv: "8b6f8492d52b28af9c4b266772f03ed0"
server_key_version: "encryptionkey@0"
I20260807 05:09:03.900210 24209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "562a6035ea9444f181e9f26a61a444fc"
format_stamp: "Formatted at 2026-08-07 05:09:03 on dist-test-slave-1x4g"
server_key: "a26ba904d7592cc2ddb0cc856367723f"
server_key_iv: "8b6f8492d52b28af9c4b266772f03ed0"
server_key_version: "encryptionkey@0"
I20260807 05:09:03.903816 24209 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 05:09:03.906220 24224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:03.907377 24209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 05:09:03.907517 24209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "562a6035ea9444f181e9f26a61a444fc"
format_stamp: "Formatted at 2026-08-07 05:09:03 on dist-test-slave-1x4g"
server_key: "a26ba904d7592cc2ddb0cc856367723f"
server_key_iv: "8b6f8492d52b28af9c4b266772f03ed0"
server_key_version: "encryptionkey@0"
I20260807 05:09:03.907622 24209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:03.912047 24209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:03.914898 24209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:03.915093 24209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:03.922092 24209 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:37809
I20260807 05:09:03.922148 24282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:37809 every 8 connection(s)
I20260807 05:09:03.923012 24209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:03.925336 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24209
I20260807 05:09:03.925482 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:09:03.925770 1095 external_mini_cluster.cc:1428] Setting key 8841832efd7306e8f79ae6af494d5815
I20260807 05:09:03.926007 24283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:03.931095 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:03.927076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51251 (local address 127.1.17.254:37809)
0807 05:09:03.927490 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:03.927510 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:03.927549 (+ 39us) server_negotiation.cc:407] Connection header received
0807 05:09:03.928309 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:03.928332 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:03.928692 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:03.929086 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:03.929742 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:03.929748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:03.929762 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:03.929773 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:03.929831 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:03.929857 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:03.929998 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:03.930073 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 05:09:03.930222 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":108,"threads_started":1}
I20260807 05:09:03.933301 24283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc: Bootstrap starting.
I20260807 05:09:03.933801 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:03.935544 24283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:03.936443 24283 log.cc:824] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:03.937366 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:03.935731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34431 (local address 127.1.17.254:37809)
0807 05:09:03.935857 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:03.935861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:03.936046 (+ 185us) server_negotiation.cc:407] Connection header received
0807 05:09:03.936128 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:03.936131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:03.936178 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:03.936259 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:03.936992 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:03.936999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:03.937004 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 05:09:03.937019 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:03.937042 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 05:09:03.937056 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:03.937161 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:03.937203 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 05:09:03.937244 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:03.938601 24283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc: No bootstrap required, opened a new log
I20260807 05:09:03.941962 24283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562a6035ea9444f181e9f26a61a444fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37809 } }
I20260807 05:09:03.942198 24283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 05:09:03.942272 24283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 562a6035ea9444f181e9f26a61a444fc, State: Initialized, Role: FOLLOWER
I20260807 05:09:03.942693 24283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562a6035ea9444f181e9f26a61a444fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37809 } }
I20260807 05:09:03.942880 24283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 05:09:03.942963 24283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 05:09:03.943058 24283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:03.944133 24283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562a6035ea9444f181e9f26a61a444fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37809 } }
I20260807 05:09:03.944459 24283 leader_election.cc:302] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 562a6035ea9444f181e9f26a61a444fc; no voters:
I20260807 05:09:03.944753 24283 leader_election.cc:288] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 05:09:03.944924 24295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:03.945199 24295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [term 1 LEADER]: Becoming Leader. State: Replica: 562a6035ea9444f181e9f26a61a444fc, State: Running, Role: LEADER
I20260807 05:09:03.945539 24295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562a6035ea9444f181e9f26a61a444fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37809 } }
I20260807 05:09:03.945999 24283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:03.947316 24297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 562a6035ea9444f181e9f26a61a444fc. Latest consensus state: current_term: 1 leader_uuid: "562a6035ea9444f181e9f26a61a444fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562a6035ea9444f181e9f26a61a444fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37809 } } }
I20260807 05:09:03.947352 24296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "562a6035ea9444f181e9f26a61a444fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "562a6035ea9444f181e9f26a61a444fc" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 37809 } } }
I20260807 05:09:03.947516 24296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:03.947888 24306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:03.947511 24297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:03.951227 24306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:03.956017 24306 catalog_manager.cc:1347] Generated new cluster ID: a86e5a6c17c94641969ac25285805ebd
I20260807 05:09:03.956143 24306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:03.974921 24306 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:03.975251 24306 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:03.986330 24306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 562a6035ea9444f181e9f26a61a444fc: Generated new TSK 0
I20260807 05:09:03.987051 24306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 05:09:03.992664 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24209
2026-08-07T05:09:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1421 ms)
[----------] 18 tests from AuthTokenIssuingTest (25658 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:04 dist-test-slave-1x4g krb5kdc[24319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:04 dist-test-slave-1x4g krb5kdc[24319](info): set up 2 sockets
Aug 07 05:09:04 dist-test-slave-1x4g krb5kdc[24319](info): commencing operation
krb5kdc: starting...
W20260807 05:09:05.034518 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:09:05.156291 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44603
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:44603,127.1.17.253:42851,127.1.17.252:36895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:05.265877 24332 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:09:05.266127 24332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:05.266149 24332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:05.266225 24332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:05.270044 24332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:05.270090 24332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:05.270108 24332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:05.270124 24332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:05.270182 24332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:05.275279 24332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:44603,127.1.17.253:42851,127.1.17.252:36895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:44603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:05.276234 24332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:05.277240 24332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:09:05.285624 24332 server_base.cc:1034] running on GCE node
W20260807 05:09:05.285575 24340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:05.285576 24337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:05.285560 24338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:05.286444 24332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:05.287429 24332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:05.288612 24332 hybrid_clock.cc:648] HybridClock initialized: now 1786079345288594 us; error 29 us; skew 500 ppm
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:05.291399 24332 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:09:05.292268 24332 webserver.cc:460] Webserver started at http://127.1.17.254:35811/ using document root <none> and password file <none>
I20260807 05:09:05.292861 24332 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:05.292971 24332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:05.293206 24332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:05.295126 24332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "f910c5ecacab41c69d04f95952df2f66"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "bbbd2c6210a2a78481412cf85c4bb6de"
server_key_iv: "fc6533ed72394c2ede4f5b1f6d2d1e17"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.295686 24332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "f910c5ecacab41c69d04f95952df2f66"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "bbbd2c6210a2a78481412cf85c4bb6de"
server_key_iv: "fc6533ed72394c2ede4f5b1f6d2d1e17"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.299252 24332 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 05:09:05.301492 24347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:05.302652 24332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:05.302788 24332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "f910c5ecacab41c69d04f95952df2f66"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "bbbd2c6210a2a78481412cf85c4bb6de"
server_key_iv: "fc6533ed72394c2ede4f5b1f6d2d1e17"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.302913 24332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:05.314523 24332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:05.317185 24332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:05.317382 24332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:05.324110 24332 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:44603
I20260807 05:09:05.324407 24402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:44603 every 8 connection(s)
I20260807 05:09:05.325086 24332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:05.327747 24403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:05.331451 24403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:05.332389 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24332
I20260807 05:09:05.332541 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:09:05.332880 1095 external_mini_cluster.cc:1428] Setting key 919706483a888daeab6b06d276619cf4
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.17.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.253@KRBTEST.COM" created.
I20260807 05:09:05.346544 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.333836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.1.17.254:44603)
0807 05:09:05.334239 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.334258 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.334585 (+ 327us) server_negotiation.cc:407] Connection header received
0807 05:09:05.335607 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.335613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.335782 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.336009 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.336957 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.337837 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.338808 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.339170 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.339335 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.341660 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.341666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.341675 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.341693 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.343187 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.343654 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.343659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.343661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.343768 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.344085 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.344090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.344092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.344602 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.344759 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.345406 (+ 647us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.345733 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":266,"thread_start_us":135,"threads_started":1}
I20260807 05:09:05.346549 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.333836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44603 (local address 127.0.0.1:41880)
0807 05:09:05.334420 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.334463 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.335306 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.336002 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.336007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.336491 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.336754 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.336767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.337975 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.337980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.338691 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.338700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.339322 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.339326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.339438 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.340049 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.340065 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.341518 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.343299 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.343309 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.343319 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.343545 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.343871 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.343874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.343876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.343993 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.344803 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.344810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.345049 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.345282 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.345598 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"spinlock_wait_cycles":7040,"thread_start_us":163,"threads_started":1}
I20260807 05:09:05.349232 24403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:05.350024 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.349755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52558)
0807 05:09:05.349869 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.349972 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 05:09:05.350276 24349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
I20260807 05:09:05.351195 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.351010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52568)
0807 05:09:05.351110 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.351151 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 05:09:05.351416 24403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:42851: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab.
Entry for principal kudu/127.1.17.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab.
I20260807 05:09:05.361020 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:42851
--webserver_interface=127.1.17.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab
--principal=kudu/127.1.17.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:44603,127.1.17.253:42851,127.1.17.252:36895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:09:05.400686 24403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } attempt: 1
I20260807 05:09:05.401785 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.401455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52574)
0807 05:09:05.401638 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.401705 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 05:09:05.402566 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.402326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52576)
0807 05:09:05.402449 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.402504 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 05:09:05.402695 24403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:42851: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
I20260807 05:09:05.470856 24403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } attempt: 2
I20260807 05:09:05.471760 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.471436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52590)
0807 05:09:05.471609 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.471671 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 05:09:05.472543 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.472326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52606)
0807 05:09:05.472438 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.472488 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 05:09:05.472676 24403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:42851: Network error: Client connection negotiation failed: client connection to 127.1.17.253:42851: connect: Connection refused (error 111)
W20260807 05:09:05.475010 24410 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:09:05.475303 24410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:05.475361 24410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:05.475436 24410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:05.479408 24410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:05.479483 24410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:05.479528 24410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:05.479584 24410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:05.479620 24410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:05.484965 24410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:44603,127.1.17.253:42851,127.1.17.252:36895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab
--principal=kudu/127.1.17.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:42851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.17.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:05.486027 24410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:05.487042 24410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:05.495640 24419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:05.495668 24416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:05.495841 24410 server_base.cc:1034] running on GCE node
W20260807 05:09:05.495992 24417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:05.496585 24410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:05.497713 24410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:05.498880 24410 hybrid_clock.cc:648] HybridClock initialized: now 1786079345498866 us; error 27 us; skew 500 ppm
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:05.501709 24410 init.cc:377] Logged in from keytab as kudu/127.1.17.253@KRBTEST.COM (short username kudu)
I20260807 05:09:05.502580 24410 webserver.cc:460] Webserver started at http://127.1.17.253:42327/ using document root <none> and password file <none>
I20260807 05:09:05.503132 24410 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:05.503240 24410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:05.503480 24410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:05.505587 24410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data/instance:
uuid: "cf014f1ebcce4ba9888fcba328c96be0"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "41d4a44ced422ab45ec1d571f104e2a8"
server_key_iv: "b410a627edd42c5fb4d9cea80c92c009"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.506140 24410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/wal/instance:
uuid: "cf014f1ebcce4ba9888fcba328c96be0"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "41d4a44ced422ab45ec1d571f104e2a8"
server_key_iv: "b410a627edd42c5fb4d9cea80c92c009"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.510056 24410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 05:09:05.512403 24426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:05.513692 24410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:05.513835 24410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/wal
uuid: "cf014f1ebcce4ba9888fcba328c96be0"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "41d4a44ced422ab45ec1d571f104e2a8"
server_key_iv: "b410a627edd42c5fb4d9cea80c92c009"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.513957 24410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:05.530794 24410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:05.534157 24410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:05.534381 24410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:05.542336 24410 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.253:42851
I20260807 05:09:05.542614 24483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.253:42851 every 8 connection(s)
I20260807 05:09:05.543370 24410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
I20260807 05:09:05.546119 24484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:05.547669 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24410
I20260807 05:09:05.547816 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-1/wal/instance
I20260807 05:09:05.548090 1095 external_mini_cluster.cc:1428] Setting key 6bfe8e66c768009e74ebff5bdb2ec882
I20260807 05:09:05.550020 24484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.17.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.252@KRBTEST.COM" created.
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:09:05.563858 24403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } attempt: 3
I20260807 05:09:05.563992 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.551874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.1.17.254:44603)
0807 05:09:05.552052 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.552059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.552855 (+ 796us) server_negotiation.cc:407] Connection header received
0807 05:09:05.553847 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.553851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.553925 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.554024 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.555153 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.555688 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.556940 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.557187 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.557305 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.560090 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.560093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.560095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.560120 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.561788 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.562347 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.562352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.562354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.562439 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.562787 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.562792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.562793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.562993 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.563103 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.563631 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.563761 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:09:05.564989 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.552212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44603 (local address 127.0.0.1:41890)
0807 05:09:05.552684 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.552736 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.553639 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.554191 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.554198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.554539 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.555004 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.555016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.555819 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.555824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.556809 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.556819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.557352 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.557357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.557540 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.558088 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.558105 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.559912 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.561917 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.561923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.561961 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.562237 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.562541 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.562545 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.562547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.562688 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.563103 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.563111 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.563379 (+ 268us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.563575 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.563941 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":156,"threads_started":1}
I20260807 05:09:05.566675 24484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab.
Entry for principal kudu/127.1.17.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab.
I20260807 05:09:05.576323 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:36895
--webserver_interface=127.1.17.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab
--principal=kudu/127.1.17.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:36541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:44603,127.1.17.253:42851,127.1.17.252:36895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:09:05.577234 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.567203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52622)
0807 05:09:05.567283 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.567296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.567368 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.567874 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.567877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.568077 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.568234 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.568241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.569345 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.569348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.570281 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.570288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.570797 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.570800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.570969 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.571452 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.571488 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.572763 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.575446 (+ 2683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.575453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.575470 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.575739 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.576023 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.576027 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.576029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.576106 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.576637 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.576658 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.576746 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.576891 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.577044 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":67}
I20260807 05:09:05.577277 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.567330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.1.17.253:42851)
0807 05:09:05.567641 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.567648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.567657 (+ 9us) server_negotiation.cc:407] Connection header received
0807 05:09:05.567709 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.567711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.567750 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.568343 (+ 593us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.568429 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.569248 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.570415 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.570689 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.570792 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.572945 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.572952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.572958 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.572988 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.575294 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.575837 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.575842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.575844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.575926 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.576200 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.576204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.576207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.576525 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.576694 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.577000 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.577131 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":254,"thread_start_us":151,"threads_started":1}
I20260807 05:09:05.577536 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.564301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52612)
0807 05:09:05.564517 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.564536 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.564632 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.565829 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.565833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.566014 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.566153 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.566161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.567742 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.567745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.569204 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.569215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.570092 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.570099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.570255 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.570878 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.570894 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.572226 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.575429 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.575432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.575434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.575645 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.576225 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.576228 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.576230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.576317 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.576886 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.576892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.577029 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.577126 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.577287 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 05:09:05.578241 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.564927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.1.17.253:42851)
0807 05:09:05.565286 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.565301 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.565329 (+ 28us) server_negotiation.cc:407] Connection header received
0807 05:09:05.565380 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.565389 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.565630 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.565834 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.566290 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.567588 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.569342 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.569969 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.570078 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.572400 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.572413 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.572430 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.572469 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.575313 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.575809 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.575815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.575817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.575908 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.576434 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.576441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.576444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.576724 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.577737 (+ 1013us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.577820 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.577980 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":229,"thread_start_us":105,"threads_started":1}
I20260807 05:09:05.579231 24403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:05.579330 24484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:05.580067 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.579714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:50978)
0807 05:09:05.579902 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.579995 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260807 05:09:05.580283 24427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
I20260807 05:09:05.581013 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.579697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:50968)
0807 05:09:05.580898 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.580957 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1128}
I20260807 05:09:05.581257 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.580975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:50992)
0807 05:09:05.581127 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.581183 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 05:09:05.581430 24484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:36895: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
I20260807 05:09:05.581748 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.581540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51008)
0807 05:09:05.581656 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.581698 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 05:09:05.581883 24403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:36895: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
I20260807 05:09:05.613055 24403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } attempt: 1
I20260807 05:09:05.614053 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.613697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51020)
0807 05:09:05.613891 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.613966 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 05:09:05.614869 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.614621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51022)
0807 05:09:05.614738 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.614796 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 05:09:05.615016 24403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:36895: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
I20260807 05:09:05.630770 24484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } attempt: 1
I20260807 05:09:05.631582 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.631309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51030)
0807 05:09:05.631451 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.631509 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 05:09:05.632252 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.632054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51040)
0807 05:09:05.632157 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.632205 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 05:09:05.632375 24484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:36895: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
I20260807 05:09:05.690186 24403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } attempt: 2
I20260807 05:09:05.691043 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.690733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51042)
0807 05:09:05.690881 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.690968 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 05:09:05.691179 24491 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:09:05.691571 24491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:05.691648 24491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:05.691740 24491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 05:09:05.691881 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.691615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51046)
0807 05:09:05.691754 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.691813 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 05:09:05.692051 24403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:36895: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
W20260807 05:09:05.695910 24491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:05.695993 24491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:05.696041 24491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:05.696076 24491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:05.696107 24491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:05.700519 24484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } attempt: 2
I20260807 05:09:05.701318 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.701053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51056)
0807 05:09:05.701182 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.701239 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 05:09:05.702025 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.701803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51064)
0807 05:09:05.701914 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.701974 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 05:09:05.702152 24484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:36895: Network error: Client connection negotiation failed: client connection to 127.1.17.252:36895: connect: Connection refused (error 111)
I20260807 05:09:05.702265 24491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:36541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:44603,127.1.17.253:42851,127.1.17.252:36895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab
--principal=kudu/127.1.17.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:36895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.17.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:05.703346 24491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:05.704387 24491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:05.712904 24498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:05.712944 24500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:05.712944 24497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:05.713076 24491 server_base.cc:1034] running on GCE node
I20260807 05:09:05.713825 24491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:05.714957 24491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:05.716120 24491 hybrid_clock.cc:648] HybridClock initialized: now 1786079345716103 us; error 24 us; skew 500 ppm
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:05.719074 24491 init.cc:377] Logged in from keytab as kudu/127.1.17.252@KRBTEST.COM (short username kudu)
I20260807 05:09:05.719926 24491 webserver.cc:460] Webserver started at http://127.1.17.252:36245/ using document root <none> and password file <none>
I20260807 05:09:05.720564 24491 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:05.720685 24491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:05.720989 24491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:05.723027 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data/instance:
uuid: "767b1c75294444919fbb8dad590563a1"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "c3e705cfcb32e009dad0311bacc8b269"
server_key_iv: "3eaaba25aafb0fce108bd6dfd5a52dee"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.723599 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/wal/instance:
uuid: "767b1c75294444919fbb8dad590563a1"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "c3e705cfcb32e009dad0311bacc8b269"
server_key_iv: "3eaaba25aafb0fce108bd6dfd5a52dee"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.727438 24491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:09:05.729888 24507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:05.731139 24491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:05.731287 24491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/wal
uuid: "767b1c75294444919fbb8dad590563a1"
format_stamp: "Formatted at 2026-08-07 05:09:05 on dist-test-slave-1x4g"
server_key: "c3e705cfcb32e009dad0311bacc8b269"
server_key_iv: "3eaaba25aafb0fce108bd6dfd5a52dee"
server_key_version: "encryptionkey@0"
I20260807 05:09:05.731407 24491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:05.749094 24491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:05.752305 24491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:05.752508 24491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:05.759322 24491 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.252:36895
I20260807 05:09:05.759608 24562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.252:36895 every 8 connection(s)
I20260807 05:09:05.760241 24491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
I20260807 05:09:05.762996 24563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:05.764873 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24491
I20260807 05:09:05.765017 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786079068077062-1095-0/minicluster-data/master-2/wal/instance
I20260807 05:09:05.765295 1095 external_mini_cluster.cc:1428] Setting key e9cd2fe5e118ca23f0fa1b3186e29843
I20260807 05:09:05.767485 24563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:09:05.779417 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.769689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.1.17.254:44603)
0807 05:09:05.769836 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.769841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.770118 (+ 277us) server_negotiation.cc:407] Connection header received
0807 05:09:05.770252 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.770255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.770301 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.770392 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.771045 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.771566 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.772457 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.772769 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.772902 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.775645 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.775656 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.775660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.775692 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.777329 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.777838 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.777841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.777843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.777969 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.778331 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.778335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.778337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.778577 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.778683 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.779120 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.779244 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 05:09:05.780143 24567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.769554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:44603 (local address 127.0.0.1:41900)
0807 05:09:05.769993 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.770039 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.770149 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.770466 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.770470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.770780 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.770890 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.770896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.771698 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.771706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.772311 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.772320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.772952 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.772962 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.773042 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.773588 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.773604 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.775477 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.777428 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.777437 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.777447 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.777745 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.778072 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.778075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.778077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.778217 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.778690 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.778695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.778857 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.779059 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.779440 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":154,"threads_started":1}
I20260807 05:09:05.780145 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.766553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.1.17.252:36895)
0807 05:09:05.767013 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.767034 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.767077 (+ 43us) server_negotiation.cc:407] Connection header received
0807 05:09:05.767881 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.767904 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.768311 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.768652 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.769391 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.770670 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.771676 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.772156 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.772335 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.775027 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.775037 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.775052 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.775084 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.777198 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.777715 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.777719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.777720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.777861 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.778205 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.778208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.778209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.778690 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.778890 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.779120 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.779318 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":136,"threads_started":1}
I20260807 05:09:05.781330 24563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:09:05.791198 24403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } attempt: 3
I20260807 05:09:05.791311 24567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.781797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:42851 (local address 127.0.0.1:52624)
0807 05:09:05.782163 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.782173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.782279 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.782652 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.782658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.782849 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.782987 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.782996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.783927 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.783930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.784672 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.784682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.785242 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.785245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.785314 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.785751 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.785766 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.787632 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.789627 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.789629 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.789630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.789922 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.790213 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.790216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.790217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.790290 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.790806 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.790812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.790906 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.791030 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.791165 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314}
I20260807 05:09:05.791534 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.781963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.1.17.253:42851)
0807 05:09:05.782095 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.782098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.782247 (+ 149us) server_negotiation.cc:407] Connection header received
0807 05:09:05.782368 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.782374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.782432 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.782522 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.783118 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.783801 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.784848 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.785124 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.785220 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.787779 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.787784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.787787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.787812 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.789500 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.790022 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.790025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.790026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.790099 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.790406 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.790409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.790410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.790640 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.790756 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.791116 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.791233 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:09:05.791971 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.782451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.1.17.254:44603)
0807 05:09:05.782556 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.782562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.782579 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:09:05.782680 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.782682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.782713 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.782771 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.783433 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.784070 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.785087 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.785387 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.785512 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.787814 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.787818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.787821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.787844 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.789719 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.790181 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.790184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.790186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.790272 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.790555 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.790558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.790559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.790772 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.790987 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.791124 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.791830 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:05.792243 24563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:05.793287 24484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } attempt: 3
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
I20260807 05:09:05.800704 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.791558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51082)
0807 05:09:05.791722 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.791736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.791806 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.792008 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.792010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.792205 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.792296 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.792302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.793217 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.793224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.794019 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.794030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.794684 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.794687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.794765 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.795211 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.795225 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.796583 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.798892 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.798895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.798897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.799147 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.799497 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.799503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.799506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.799607 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.800113 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.800119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.800247 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.800340 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.800505 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 05:09:05.801482 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.791613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.1.17.252:36895)
0807 05:09:05.791810 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.791813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.791827 (+ 14us) server_negotiation.cc:407] Connection header received
0807 05:09:05.791876 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.791889 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.791928 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.791992 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.792434 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.793082 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.794178 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.794563 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.794966 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.796746 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.796750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.796753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.796794 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.798760 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.799273 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.799277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.799279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.799373 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.799738 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.799744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.799748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.799986 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.800928 (+ 942us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.801183 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.801290 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":136}
I20260807 05:09:05.803126 24567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.792673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51098)
0807 05:09:05.792871 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.792884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.793101 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.793573 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.793575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.793728 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.793791 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.793796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.795147 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.795150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.795944 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.795956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.796855 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.796858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.797042 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.797447 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.797466 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.798927 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.801322 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.801324 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.801327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.801852 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.802183 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.802185 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.802187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.802268 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.802670 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.802672 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.802767 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.802879 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.802993 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":40}
Aug 07 05:09:05 dist-test-slave-1x4g krb5kdc[24319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
I20260807 05:09:05.803853 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.793598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:36895 (local address 127.0.0.1:51110)
0807 05:09:05.793728 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:05.793739 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:05.793809 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:05.794487 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:05.794492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:05.794684 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:05.794755 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.794760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.795677 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.795682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.796491 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:05.796499 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.797741 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.797745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:05.797826 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.798276 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:05.798289 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:05.799735 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:05.802208 (+ 2473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.802212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.802217 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.802494 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.802829 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:05.802835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:05.802838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:05.802942 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:05.803379 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:05.803383 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:05.803477 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 05:09:05.803562 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 05:09:05.803694 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 05:09:05.803714 24568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.793022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.1.17.252:36895)
0807 05:09:05.793321 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.793330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.793341 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:09:05.793371 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.793373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.793409 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.793489 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.793899 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.795043 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.796074 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.796499 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.796680 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.799072 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.799077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.799080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.799105 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.801216 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.801958 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.801962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.801966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.802085 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.802359 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.802362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.802363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.802543 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.802634 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.803431 (+ 797us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.803546 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":95,"threads_started":1}
I20260807 05:09:05.805069 24403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66: Bootstrap starting.
I20260807 05:09:05.805121 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.794042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.1.17.252:36895)
0807 05:09:05.794283 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.794290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.794300 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:09:05.794327 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.794328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.794356 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.794467 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.794862 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.795560 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.796617 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.797595 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.798039 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.799877 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.799892 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.799894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.799917 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.802087 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.802621 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.802624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.802626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.802710 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.803065 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.803068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.803070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.803261 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.804722 (+ 1461us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.804817 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.804951 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":477,"server-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
I20260807 05:09:05.806746 24563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: Bootstrap starting.
I20260807 05:09:05.807792 24484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: Bootstrap starting.
I20260807 05:09:05.807886 24403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:05.808559 24403 log.cc:824] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:05.809152 24563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:05.809959 24563 log.cc:824] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:05.810168 24484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:05.810765 24403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66: No bootstrap required, opened a new log
I20260807 05:09:05.810853 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.793594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.1.17.253:42851)
0807 05:09:05.793694 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.793698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.796749 (+ 3051us) server_negotiation.cc:407] Connection header received
0807 05:09:05.796939 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.796943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.796991 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.797052 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.797945 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.798662 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.799814 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.800147 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.800217 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.803917 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.803920 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.803924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.803949 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.805692 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.809401 (+ 3709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.809404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.809406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.809495 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.809804 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.809808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.809809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.810036 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.810218 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.810604 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.810725 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:05.811230 24484 log.cc:824] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:05.813156 24563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: No bootstrap required, opened a new log
I20260807 05:09:05.813556 24484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: No bootstrap required, opened a new log
I20260807 05:09:05.815016 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:05.817404 24373 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f910c5ecacab41c69d04f95952df2f66"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41906
I20260807 05:09:05.817451 24484 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.817687 24484 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf014f1ebcce4ba9888fcba328c96be0, State: Initialized, Role: FOLLOWER
I20260807 05:09:05.818039 24484 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.818056 24563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.818289 24563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 767b1c75294444919fbb8dad590563a1, State: Initialized, Role: FOLLOWER
I20260807 05:09:05.818562 24573 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.818578 24563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.818738 24573 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:05.814890 24403 raft_consensus.cc:348] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.818873 24403 raft_consensus.cc:729] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f910c5ecacab41c69d04f95952df2f66, State: Initialized, Role: FOLLOWER
I20260807 05:09:05.819231 24403 consensus_queue.cc:260] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.820014 24484 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:05.820014 24563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:05.820464 24403 sys_catalog.cc:564] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:05.820513 24373 raft_consensus.cc:482] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 05:09:05.820606 24373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:05.822095 24373 raft_consensus.cc:504] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.823112 24373 leader_election.cc:288] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [CANDIDATE]: Term 1 election: Requested vote from peers cf014f1ebcce4ba9888fcba328c96be0 (127.1.17.253:42851), 767b1c75294444919fbb8dad590563a1 (127.1.17.252:36895)
I20260807 05:09:05.824023 24455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f910c5ecacab41c69d04f95952df2f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cf014f1ebcce4ba9888fcba328c96be0"
I20260807 05:09:05.824036 24536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f910c5ecacab41c69d04f95952df2f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "767b1c75294444919fbb8dad590563a1"
I20260807 05:09:05.824249 24455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:05.824268 24536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:05.825662 24536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f910c5ecacab41c69d04f95952df2f66 in term 1.
I20260807 05:09:05.825699 24455 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f910c5ecacab41c69d04f95952df2f66 in term 1.
W20260807 05:09:05.825814 24605 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:05.825906 24605 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:05.826152 24349 leader_election.cc:302] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf014f1ebcce4ba9888fcba328c96be0, f910c5ecacab41c69d04f95952df2f66; no voters:
I20260807 05:09:05.826722 24614 sys_catalog.cc:455] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.827075 24614 sys_catalog.cc:458] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:05.827176 24614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:05.827250 24614 raft_consensus.cc:686] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [term 1 LEADER]: Becoming Leader. State: Replica: f910c5ecacab41c69d04f95952df2f66, State: Running, Role: LEADER
I20260807 05:09:05.827554 24614 consensus_queue.cc:237] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } }
I20260807 05:09:05.825870 24491 master_runner.cc:186] Error getting master registration for 127.1.17.254:44603: OK, instance_id { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" instance_seqno: 1786079345322303 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:09:05.829231 24491 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:09:05.829481 24614 sys_catalog.cc:455] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f910c5ecacab41c69d04f95952df2f66. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.829608 24614 sys_catalog.cc:458] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:05.830250 24622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:05.826984 24410 master_runner.cc:186] Error getting master registration for 127.1.17.254:44603: OK, instance_id { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" instance_seqno: 1786079345322303 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:09:05.830399 24410 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 05:09:05.833302 24604 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:05.833395 24604 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:05.833400 24622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:05.839351 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.825455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.1.17.253:42851)
0807 05:09:05.825584 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.825591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.829971 (+ 4380us) server_negotiation.cc:407] Connection header received
0807 05:09:05.830274 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.830280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.830333 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.830385 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.831501 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.832189 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.833397 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.833716 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.833797 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.835859 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.835875 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.835879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.835909 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.837557 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.838004 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.838008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.838010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.838096 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.838384 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.838387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.838389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.838617 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.838737 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.839084 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.839210 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:05.839596 24536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [term 1 FOLLOWER]: Refusing update from remote peer f910c5ecacab41c69d04f95952df2f66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:05.839636 24455 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [term 1 FOLLOWER]: Refusing update from remote peer f910c5ecacab41c69d04f95952df2f66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:05.840343 24614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:05.840608 24575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:05.844303 24573 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f910c5ecacab41c69d04f95952df2f66. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.844467 24573 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:05.848905 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.826414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.1.17.254:44603)
0807 05:09:05.826562 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.826566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.840962 (+ 14396us) server_negotiation.cc:407] Connection header received
0807 05:09:05.841135 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.841138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.841186 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.841239 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.842088 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.842803 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.843885 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.844214 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.844296 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.845730 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.845733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.845736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.845759 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.847177 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.847650 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.847655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.847657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.847743 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.848025 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.848029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.848030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.848209 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.848360 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.848576 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.848683 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 05:09:05.852969 24574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.853142 24574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:05.853263 24574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f910c5ecacab41c69d04f95952df2f66. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.853353 24574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:05.857386 24625 mvcc.cc:205] Tried to move back new op lower bound from 7315781000552587264 to 7315781000513323008. Current Snapshot: MvccSnapshot[applied={T|T < 7315781000552587264}]
I20260807 05:09:05.857479 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:05.827099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.1.17.252:36895)
0807 05:09:05.827227 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.827231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.849250 (+ 22019us) server_negotiation.cc:407] Connection header received
0807 05:09:05.849424 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.849427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.849476 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.849530 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.850261 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.850746 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.851474 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:05.851743 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:05.851848 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:05.853631 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:05.853634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:05.853637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:05.853662 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:05.855267 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.855745 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.855748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.855750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.855833 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:05.856107 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:05.856112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:05.856117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:05.856352 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:05.856493 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:05.857166 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 05:09:05.857298 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 05:09:05.858603 24575 sys_catalog.cc:455] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.858736 24575 sys_catalog.cc:458] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:05.858846 24575 sys_catalog.cc:455] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.858917 24575 sys_catalog.cc:458] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:05.860607 24573 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.860718 24573 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:05.861457 24627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f910c5ecacab41c69d04f95952df2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910c5ecacab41c69d04f95952df2f66" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 44603 } } peers { permanent_uuid: "cf014f1ebcce4ba9888fcba328c96be0" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 42851 } } peers { permanent_uuid: "767b1c75294444919fbb8dad590563a1" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 36895 } } }
I20260807 05:09:05.861598 24627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:05.862623 24622 catalog_manager.cc:1347] Generated new cluster ID: 93ae69f43c6d454fb3bbd04c7b143d53
I20260807 05:09:05.862681 24622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:05.862931 24629 mvcc.cc:205] Tried to move back new op lower bound from 7315781000552587264 to 7315781000513323008. Current Snapshot: MvccSnapshot[applied={T|T < 7315781000552587264}]
I20260807 05:09:05.906396 24622 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:05.908428 24622 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:05.921655 24622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f910c5ecacab41c69d04f95952df2f66: Generated new TSK 0
I20260807 05:09:05.922554 24622 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:09:05.959048 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:05.955604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52668 (local address 127.1.17.253:42851)
0807 05:09:05.955761 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.955766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.957829 (+ 2063us) server_negotiation.cc:407] Connection header received
0807 05:09:05.958055 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.958059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.958115 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.958176 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.958894 (+ 718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52668: BlockingRecv error: recv got EOF from 127.0.0.1:52668 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 05:09:05.960589 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:05.956482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.1.17.254:44603)
0807 05:09:05.956622 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.956630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.959602 (+ 2972us) server_negotiation.cc:407] Connection header received
0807 05:09:05.959791 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.959795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.959848 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.959917 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.960456 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41928: BlockingRecv error: recv got EOF from 127.0.0.1:41928 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 05:09:05.962093 24569 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:05.957204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.1.17.252:36895)
0807 05:09:05.957325 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.957330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.961132 (+ 3802us) server_negotiation.cc:407] Connection header received
0807 05:09:05.961330 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.961333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.961397 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.961454 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.961966 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51136: BlockingRecv error: recv got EOF from 127.0.0.1:51136 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 05:09:05.970681 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:05.969256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.1.17.253:42851)
0807 05:09:05.969385 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.969389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.969631 (+ 242us) server_negotiation.cc:407] Connection header received
0807 05:09:05.969869 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.969872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.969957 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.970015 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.970566 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52670: BlockingRecv error: recv got EOF from 127.0.0.1:52670 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 05:09:05.981828 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:05.980206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.1.17.253:42851)
0807 05:09:05.980314 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.980318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.980865 (+ 547us) server_negotiation.cc:407] Connection header received
0807 05:09:05.981062 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.981066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.981118 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.981172 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.981722 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52672: BlockingRecv error: recv got EOF from 127.0.0.1:52672 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 05:09:05.987569 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:05.986056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.1.17.253:42851)
0807 05:09:05.986155 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.986159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.986672 (+ 513us) server_negotiation.cc:407] Connection header received
0807 05:09:05.986855 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.986859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.986909 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.986962 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.987470 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52674: BlockingRecv error: recv got EOF from 127.0.0.1:52674 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 05:09:05.995514 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:05.993863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.1.17.253:42851)
0807 05:09:05.993995 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:05.993999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:05.994559 (+ 560us) server_negotiation.cc:407] Connection header received
0807 05:09:05.994772 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:05.994776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:05.994827 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:05.994883 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:05.995416 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52678: BlockingRecv error: recv got EOF from 127.0.0.1:52678 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 05:09:06.008410 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:06.006511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52694 (local address 127.1.17.253:42851)
0807 05:09:06.006629 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:06.006634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:06.007300 (+ 666us) server_negotiation.cc:407] Connection header received
0807 05:09:06.007536 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:06.007541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:06.007600 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:06.007661 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:06.008255 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52694: BlockingRecv error: recv got EOF from 127.0.0.1:52694 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 05:09:06.028955 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:06.027079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.1.17.253:42851)
0807 05:09:06.027198 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:06.027203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:06.027825 (+ 622us) server_negotiation.cc:407] Connection header received
0807 05:09:06.028057 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:06.028061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:06.028118 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:06.028178 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:06.028830 (+ 652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52710: BlockingRecv error: recv got EOF from 127.0.0.1:52710 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 05:09:06.065371 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:06.063628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52724 (local address 127.1.17.253:42851)
0807 05:09:06.063774 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:06.063779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:06.064072 (+ 293us) server_negotiation.cc:407] Connection header received
0807 05:09:06.064279 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:06.064285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:06.064372 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:06.064443 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:06.065219 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52724: BlockingRecv error: recv got EOF from 127.0.0.1:52724 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 05:09:06.133381 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:06.132107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52736 (local address 127.1.17.253:42851)
0807 05:09:06.132262 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:06.132266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:06.132449 (+ 183us) server_negotiation.cc:407] Connection header received
0807 05:09:06.132576 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:06.132579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:06.132626 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:06.132700 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:06.133218 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52736: BlockingRecv error: recv got EOF from 127.0.0.1:52736 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 05:09:06.264923 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:06.263586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.1.17.253:42851)
0807 05:09:06.263738 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:06.263743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:06.263865 (+ 122us) server_negotiation.cc:407] Connection header received
0807 05:09:06.264059 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:06.264064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:06.264122 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:06.264217 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:06.264731 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52744: BlockingRecv error: recv got EOF from 127.0.0.1:52744 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 05:09:06.524360 24489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:06.523130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.1.17.253:42851)
0807 05:09:06.523315 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:06.523319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:06.523413 (+ 94us) server_negotiation.cc:407] Connection header received
0807 05:09:06.523603 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:06.523607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:06.523666 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:06.523760 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:06.524244 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52756: BlockingRecv error: recv got EOF from 127.0.0.1:52756 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 05:09:06.827440 24605 catalog_manager.cc:1259] Loaded cluster ID: 93ae69f43c6d454fb3bbd04c7b143d53
I20260807 05:09:06.827548 24605 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: loading cluster ID for follower catalog manager: success
I20260807 05:09:06.829366 24605 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: acquiring CA information for follower catalog manager: success
I20260807 05:09:06.831240 24605 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 767b1c75294444919fbb8dad590563a1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:09:06.834775 24604 catalog_manager.cc:1259] Loaded cluster ID: 93ae69f43c6d454fb3bbd04c7b143d53
I20260807 05:09:06.834843 24604 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: loading cluster ID for follower catalog manager: success
I20260807 05:09:06.836894 24604 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: acquiring CA information for follower catalog manager: success
I20260807 05:09:06.837929 24604 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf014f1ebcce4ba9888fcba328c96be0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:09:07.042567 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:07.038651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.1.17.253:42851)
0807 05:09:07.038891 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:07.038895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:07.038910 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:09:07.039057 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:07.039060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:07.039192 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:07.039288 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:07.039531 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.040048 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.040967 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.041212 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.041325 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:07.041603 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:07.042066 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:07.042192 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:07.042264 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 05:09:07.042317 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260807 05:09:07.049201 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:07.045856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.1.17.252:36895)
0807 05:09:07.046094 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:07.046097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:07.046112 (+ 15us) server_negotiation.cc:407] Connection header received
0807 05:09:07.046212 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:07.046214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:07.046324 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:07.046434 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:07.046608 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.047156 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.047932 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.048144 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.048227 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:07.048463 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:07.048882 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:07.048977 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:07.049035 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 05:09:07.049082 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260807 05:09:07.056624 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:07.052075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52784 (local address 127.1.17.253:42851)
0807 05:09:07.052201 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:07.052207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:07.052618 (+ 411us) server_negotiation.cc:407] Connection header received
0807 05:09:07.052808 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:07.052813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:07.052890 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:07.052977 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:07.053205 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.053955 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.055189 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.055550 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.055653 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:07.056125 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:07.056201 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:07.056271 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:07.056416 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 05:09:07.056467 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:07.056739 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:07.052492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.1.17.252:36895)
0807 05:09:07.052609 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:07.052616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:07.052680 (+ 64us) server_negotiation.cc:407] Connection header received
0807 05:09:07.052893 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:07.052896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:07.052951 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:07.053024 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:07.053432 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.054151 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.055328 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.055563 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.055687 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:07.056170 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:07.056271 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:07.056360 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:07.056517 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 05:09:07.056581 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 05:09:07.057905 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:07.052132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.1.17.254:44603)
0807 05:09:07.052394 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:07.052401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:07.052579 (+ 178us) server_negotiation.cc:407] Connection header received
0807 05:09:07.052683 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:07.052686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:07.052901 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:07.053017 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:07.054263 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.054767 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.056146 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:07.056490 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:07.056617 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:07.057122 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:07.057448 (+ 326us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:07.057549 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:07.057683 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 05:09:07.057742 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260807 05:09:07.060940 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24332
I20260807 05:09:07.069732 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24410
I20260807 05:09:07.076586 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24491
2026-08-07T05:09:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3083 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 05:09:07 dist-test-slave-1x4g krb5kdc[24719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 05:09:07 dist-test-slave-1x4g krb5kdc[24719](info): set up 2 sockets
Aug 07 05:09:07 dist-test-slave-1x4g krb5kdc[24719](info): commencing operation
krb5kdc: starting...
W20260807 05:09:08.142511 1095 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/test-user.keytab.
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T05:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T05:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.17.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
Entry for principal kudu/127.1.17.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab.
I20260807 05:09:08.265576 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39391
--webserver_interface=127.1.17.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:39391,127.1.17.253:37471,127.1.17.252:39043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 05:09:08.376048 24733 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:09:08.376313 24733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:08.376336 24733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:08.376425 24733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:08.380445 24733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:08.380492 24733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:08.380512 24733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:08.380553 24733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:08.380585 24733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:08.385571 24733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:39391,127.1.17.253:37471,127.1.17.252:39043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.254.keytab
--principal=kudu/127.1.17.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.254:39391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.17.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:08.386480 24733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:08.387408 24733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:08.395628 24741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:08.395635 24739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:08.395628 24738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:08.395941 24733 server_base.cc:1034] running on GCE node
I20260807 05:09:08.396607 24733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:08.397747 24733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:08.398911 24733 hybrid_clock.cc:648] HybridClock initialized: now 1786079348398891 us; error 30 us; skew 500 ppm
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:08.406652 24733 init.cc:377] Logged in from keytab as kudu/127.1.17.254@KRBTEST.COM (short username kudu)
I20260807 05:09:08.407580 24733 webserver.cc:460] Webserver started at http://127.1.17.254:33853/ using document root <none> and password file <none>
I20260807 05:09:08.408133 24733 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:08.408192 24733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:08.408430 24733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:08.410562 24733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data/instance:
uuid: "c2382e3a70fe4014b8ee5ad5adb94f76"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "33c91f145b0242dc9569f48982874f6f"
server_key_iv: "c5bcae51f0c315459a7a6bf289d311ed"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.411185 24733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/wal/instance:
uuid: "c2382e3a70fe4014b8ee5ad5adb94f76"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "33c91f145b0242dc9569f48982874f6f"
server_key_iv: "c5bcae51f0c315459a7a6bf289d311ed"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.415252 24733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 05:09:08.417587 24748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:08.418738 24733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 05:09:08.418879 24733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/wal
uuid: "c2382e3a70fe4014b8ee5ad5adb94f76"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "33c91f145b0242dc9569f48982874f6f"
server_key_iv: "c5bcae51f0c315459a7a6bf289d311ed"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.418989 24733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:08.434551 24733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:08.437397 24733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:08.437597 24733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:08.444687 24733 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.254:39391
I20260807 05:09:08.444972 24803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.254:39391 every 8 connection(s)
I20260807 05:09:08.445711 24733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/data/info.pb
I20260807 05:09:08.448385 24804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:08.451445 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24733
I20260807 05:09:08.451565 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-0/wal/instance
I20260807 05:09:08.451843 1095 external_mini_cluster.cc:1428] Setting key 19e3353e712868f6bf43dea3a8ad6545
I20260807 05:09:08.452184 24804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.17.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.253@KRBTEST.COM" created.
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:09:08.469424 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.454905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39391 (local address 127.0.0.1:38116)
0807 05:09:08.455457 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.455492 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.456291 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.457040 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.457045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.457482 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.457725 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.457738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.458864 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.458867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.459621 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.459630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.460258 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.460262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.460449 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.461144 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.461160 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.464368 (+ 3208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.466369 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.466377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.466387 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.466613 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.466947 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.466950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.466951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.467065 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.467649 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.467654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.467794 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.467968 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.468353 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":179,"spinlock_wait_cycles":6400,"thread_start_us":166,"threads_started":1}
I20260807 05:09:08.469429 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.454904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.1.17.254:39391)
0807 05:09:08.455308 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.455327 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.455593 (+ 266us) server_negotiation.cc:407] Connection header received
0807 05:09:08.456587 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.456591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.456853 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.457041 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.457887 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.458740 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.459742 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.460107 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.460337 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.464518 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.464529 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.464539 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.464559 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.466207 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.466727 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.466739 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.466740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.466838 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.467159 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.467164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.467166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.467538 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.467676 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.468030 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.468325 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":260,"thread_start_us":127,"threads_started":1}
I20260807 05:09:08.471661 24804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:08.472430 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.472140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48862)
0807 05:09:08.472274 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.472339 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 05:09:08.472625 24752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
I20260807 05:09:08.473470 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.473259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48872)
0807 05:09:08.473357 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.473400 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 05:09:08.473626 24804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:37471: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab.
Entry for principal kudu/127.1.17.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab.
I20260807 05:09:08.479202 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:37471
--webserver_interface=127.1.17.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab
--principal=kudu/127.1.17.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:39391,127.1.17.253:37471,127.1.17.252:39043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:09:08.522842 24804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } attempt: 1
I20260807 05:09:08.523797 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.523449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48882)
0807 05:09:08.523655 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.523718 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 05:09:08.524564 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.524330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48884)
0807 05:09:08.524454 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.524505 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 05:09:08.524708 24804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:37471: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
W20260807 05:09:08.591537 24811 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:09:08.591917 24811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:08.592082 24811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:08.592227 24811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 05:09:08.592860 24804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } attempt: 2
I20260807 05:09:08.593669 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.593362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48900)
0807 05:09:08.593529 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.593594 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 05:09:08.594372 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.594154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48902)
0807 05:09:08.594265 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.594314 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 05:09:08.594547 24804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.253:37471: Network error: Client connection negotiation failed: client connection to 127.1.17.253:37471: connect: Connection refused (error 111)
W20260807 05:09:08.596618 24811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:08.596700 24811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:08.596746 24811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:08.596817 24811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:08.596853 24811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:08.602154 24811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:39391,127.1.17.253:37471,127.1.17.252:39043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.253.keytab
--principal=kudu/127.1.17.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.253:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.17.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:08.603178 24811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:08.604207 24811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 05:09:08.613009 24817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:08.613152 24811 server_base.cc:1034] running on GCE node
W20260807 05:09:08.613087 24816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:08.612987 24819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:08.613891 24811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:08.614946 24811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:08.616125 24811 hybrid_clock.cc:648] HybridClock initialized: now 1786079348616095 us; error 42 us; skew 500 ppm
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:08.618911 24811 init.cc:377] Logged in from keytab as kudu/127.1.17.253@KRBTEST.COM (short username kudu)
I20260807 05:09:08.619772 24811 webserver.cc:460] Webserver started at http://127.1.17.253:44515/ using document root <none> and password file <none>
I20260807 05:09:08.620324 24811 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:08.620414 24811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:08.620651 24811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:08.622790 24811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data/instance:
uuid: "2e7f3642a277432198fd6d1b8cec7cf2"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "ad4195ffff2698040da6b7e6b4e11b71"
server_key_iv: "abdcd5bdea1abfc72aa10fc76fd3b164"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.623344 24811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/wal/instance:
uuid: "2e7f3642a277432198fd6d1b8cec7cf2"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "ad4195ffff2698040da6b7e6b4e11b71"
server_key_iv: "abdcd5bdea1abfc72aa10fc76fd3b164"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.627081 24811 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 05:09:08.629478 24826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:08.630716 24811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 05:09:08.630880 24811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/wal
uuid: "2e7f3642a277432198fd6d1b8cec7cf2"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "ad4195ffff2698040da6b7e6b4e11b71"
server_key_iv: "abdcd5bdea1abfc72aa10fc76fd3b164"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.631045 24811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:08.653810 24811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:08.656626 24811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:08.656917 24811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:08.663823 24811 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.253:37471
I20260807 05:09:08.664146 24879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.253:37471 every 8 connection(s)
I20260807 05:09:08.664800 24811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/data/info.pb
I20260807 05:09:08.665572 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24811
I20260807 05:09:08.665735 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-1/wal/instance
I20260807 05:09:08.666025 1095 external_mini_cluster.cc:1428] Setting key 876bbfd5d50cb22e278c9dcc9ecb315b
I20260807 05:09:08.667621 24880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:08.671339 24880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.17.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.17.252@KRBTEST.COM" created.
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
I20260807 05:09:08.685708 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.673238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38124 (local address 127.1.17.254:39391)
0807 05:09:08.673371 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.673375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.674387 (+ 1012us) server_negotiation.cc:407] Connection header received
0807 05:09:08.675378 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.675381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.675448 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.675548 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.676755 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.677404 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.678643 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.678890 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.679009 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.681839 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.681842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.681844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.681871 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.683339 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.683917 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.683924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.683927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.684030 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.684381 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.684385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.684387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.684661 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.684831 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.685334 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.685488 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 05:09:08.685730 24804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } attempt: 3
I20260807 05:09:08.686573 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.673729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39391 (local address 127.0.0.1:38124)
0807 05:09:08.674234 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.674275 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.675164 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.675711 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.675718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.676074 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.676606 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.676620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.677548 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.677551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.678515 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.678525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.679047 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.679051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.679214 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.679797 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.679813 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.681647 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.683470 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.683476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.683506 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.683800 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.684136 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.684138 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.684140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.684271 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.684819 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.684829 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.685072 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.685227 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.685475 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":149,"threads_started":1}
I20260807 05:09:08.688163 24880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.17.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab.
Entry for principal kudu/127.1.17.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab.
I20260807 05:09:08.696616 1095 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXAtKix/build/debug/bin/kudu
/tmp/dist-test-taskXAtKix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:39043
--webserver_interface=127.1.17.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab
--principal=kudu/127.1.17.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.17.212:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.17.254:39391,127.1.17.253:37471,127.1.17.252:39043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 05:09:08.697428 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.686322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48918)
0807 05:09:08.686547 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.686564 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.686645 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.687402 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.687405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.687597 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.687705 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.687712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.689024 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.689028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.690117 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.690128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.690895 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.690899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.691016 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.691460 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.691475 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.692882 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.695602 (+ 2720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.695608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.695613 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.695906 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.696212 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.696214 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.696215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.696290 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.696983 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.696986 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.697089 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.697185 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.697294 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260807 05:09:08.697825 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.686437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.1.17.253:37471)
0807 05:09:08.686922 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.686933 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.686954 (+ 21us) server_negotiation.cc:407] Connection header received
0807 05:09:08.686992 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.687016 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.687208 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.687420 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.687866 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.688720 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.690239 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.690784 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.690879 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.693063 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.693076 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.693094 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.693126 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.695427 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.696008 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.696014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.696016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.696117 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.696383 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.696390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.696393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.696753 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.696983 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.697315 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.697532 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":151,"threads_started":1}
I20260807 05:09:08.700615 24804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:08.701251 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.688667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48924)
0807 05:09:08.688768 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.688801 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.688872 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.689409 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.689414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.689576 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.689650 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.689655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.690796 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.690798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.691579 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.691593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.692074 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.692078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.692163 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.692636 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.692649 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.694040 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.699460 (+ 5420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.699464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.699466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.699724 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.700038 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.700041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.700042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.700122 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.700618 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.700623 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.700719 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.700861 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.701019 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 05:09:08.701385 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.701116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43870)
0807 05:09:08.701227 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.701278 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 05:09:08.701829 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.688833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.1.17.253:37471)
0807 05:09:08.689160 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.689173 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.689183 (+ 10us) server_negotiation.cc:407] Connection header received
0807 05:09:08.689209 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.689211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.689240 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.689388 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.689797 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.690684 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.691713 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.691954 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.692074 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.694198 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.694209 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.694211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.694233 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.699333 (+ 5100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.699831 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.699834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.699835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.699923 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.700215 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.700218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.700220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.700468 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.700582 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.701454 (+ 872us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.701579 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":116,"threads_started":1}
I20260807 05:09:08.702077 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.701820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43882)
0807 05:09:08.701970 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.702015 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 05:09:08.702194 24804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:39043: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
I20260807 05:09:08.702618 24880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:08.703292 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.703066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43886)
0807 05:09:08.703179 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.703242 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 05:09:08.703483 24830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
I20260807 05:09:08.704252 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.704055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43902)
0807 05:09:08.704156 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.704207 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 05:09:08.704411 24880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:39043: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
I20260807 05:09:08.733349 24804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } attempt: 1
I20260807 05:09:08.734193 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.733910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43912)
0807 05:09:08.734057 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.734115 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 05:09:08.734963 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.734754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43920)
0807 05:09:08.734856 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.734905 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 05:09:08.735075 24804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:39043: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
I20260807 05:09:08.753636 24880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } attempt: 1
I20260807 05:09:08.754482 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.754170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43924)
0807 05:09:08.754351 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.754408 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 05:09:08.755229 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.755005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43932)
0807 05:09:08.755120 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.755173 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 05:09:08.755379 24880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:39043: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
I20260807 05:09:08.810230 24804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } attempt: 2
I20260807 05:09:08.811069 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.810781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43948)
0807 05:09:08.810919 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.810995 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 05:09:08.811846 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.811654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43958)
0807 05:09:08.811752 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.811795 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 05:09:08.812041 24804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:39043: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
W20260807 05:09:08.813681 24887 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 05:09:08.813975 24887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 05:09:08.814026 24887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 05:09:08.814098 24887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 05:09:08.818188 24887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 05:09:08.818238 24887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 05:09:08.818280 24887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 05:09:08.818315 24887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 05:09:08.818348 24887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 05:09:08.823577 24880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } attempt: 2
I20260807 05:09:08.823873 24887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.17.212:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.17.254:39391,127.1.17.253:37471,127.1.17.252:39043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/krb5kdc/kudu_127.1.17.252.keytab
--principal=kudu/127.1.17.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.17.252:39043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.17.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 04:14:58 UTC on ccycloud.quasar-wkuilx.root.comops.site
build id 4571822
I20260807 05:09:08.824445 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.824084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43964)
0807 05:09:08.824291 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.824350 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 05:09:08.825229 24887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 05:09:08.825320 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.825073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43976)
0807 05:09:08.825187 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.825251 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 05:09:08.825457 24880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.17.252:39043: Network error: Client connection negotiation failed: client connection to 127.1.17.252:39043: connect: Connection refused (error 111)
I20260807 05:09:08.826531 24887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 05:09:08.837218 24887 server_base.cc:1034] running on GCE node
W20260807 05:09:08.837461 24894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:08.837489 24896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 05:09:08.837427 24893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 05:09:08.838235 24887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 05:09:08.839416 24887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 05:09:08.840749 24887 hybrid_clock.cc:648] HybridClock initialized: now 1786079348840735 us; error 38 us; skew 500 ppm
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 05:09:08.843742 24887 init.cc:377] Logged in from keytab as kudu/127.1.17.252@KRBTEST.COM (short username kudu)
I20260807 05:09:08.844851 24887 webserver.cc:460] Webserver started at http://127.1.17.252:35129/ using document root <none> and password file <none>
I20260807 05:09:08.845611 24887 fs_manager.cc:359] Metadata directory not provided
I20260807 05:09:08.845692 24887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 05:09:08.845968 24887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 05:09:08.848068 24887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data/instance:
uuid: "02c5e3d9df5a4c12b549ca92fc7589b5"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "cda55f5d585d068e6e8d681fa0682662"
server_key_iv: "b32e7d289f34f1b88dcdc5113219ae78"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.848595 24887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/wal/instance:
uuid: "02c5e3d9df5a4c12b549ca92fc7589b5"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "cda55f5d585d068e6e8d681fa0682662"
server_key_iv: "b32e7d289f34f1b88dcdc5113219ae78"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.852288 24887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 05:09:08.854717 24903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 05:09:08.855962 24887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 05:09:08.856072 24887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data,/tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/wal
uuid: "02c5e3d9df5a4c12b549ca92fc7589b5"
format_stamp: "Formatted at 2026-08-07 05:09:08 on dist-test-slave-1x4g"
server_key: "cda55f5d585d068e6e8d681fa0682662"
server_key_iv: "b32e7d289f34f1b88dcdc5113219ae78"
server_key_version: "encryptionkey@0"
I20260807 05:09:08.856153 24887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 05:09:08.876422 24887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 05:09:08.879652 24887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 05:09:08.879791 24887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 05:09:08.886533 24887 rpc_server.cc:307] RPC server started. Bound to: 127.1.17.252:39043
I20260807 05:09:08.886801 24958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.17.252:39043 every 8 connection(s)
I20260807 05:09:08.887456 24887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/data/info.pb
I20260807 05:09:08.890162 24959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 05:09:08.893563 24959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:08.895347 1095 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXAtKix/build/debug/bin/kudu as pid 24887
I20260807 05:09:08.895462 1095 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXAtKix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786079068077062-1095-0/minicluster-data/master-2/wal/instance
I20260807 05:09:08.895759 1095 external_mini_cluster.cc:1428] Setting key e78f757772772ca444a742358a420c48
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
I20260807 05:09:08.906762 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.895265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.1.17.254:39391)
0807 05:09:08.895388 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.895392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.896399 (+ 1007us) server_negotiation.cc:407] Connection header received
0807 05:09:08.897571 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.897574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.897621 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.897687 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.898615 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.899199 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.900289 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.900658 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.900754 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.903162 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.903165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.903167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.903187 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.904635 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.905213 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.905216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.905218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.905297 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.905597 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.905602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.905604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.905824 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.905893 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.906476 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.906610 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 05:09:08.907677 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.895692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.254:39391 (local address 127.0.0.1:38130)
0807 05:09:08.896234 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.896294 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.897343 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.897773 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.897791 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.898235 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.898470 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.898482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.899309 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.899312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.900179 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.900187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.900812 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.900817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.901010 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.901527 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.901540 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.903022 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.904759 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.904769 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.904798 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.905118 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.905378 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.905380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.905381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.905491 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.905924 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.905931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.906192 (+ 261us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.906394 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.906647 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":137,"threads_started":1}
I20260807 05:09:08.907677 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.896500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.1.17.252:39043)
0807 05:09:08.896865 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.896877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.896919 (+ 42us) server_negotiation.cc:407] Connection header received
0807 05:09:08.897344 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.897366 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.897613 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.897826 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.898705 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.899732 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.900748 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.901212 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.901309 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.903813 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.903822 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.903837 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.903858 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.905647 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.906080 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.906084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.906085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.906177 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.906462 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.906466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.906467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.906721 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.906918 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.907135 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.907236 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"spinlock_wait_cycles":7936,"thread_start_us":164,"threads_started":1}
I20260807 05:09:08.908985 24959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } has no permanent_uuid. Determining permanent_uuid...
I20260807 05:09:08.911187 24804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } attempt: 3
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.254@KRBTEST.COM
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.254@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
I20260807 05:09:08.916612 24880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } attempt: 3
I20260807 05:09:08.920127 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.910216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38132 (local address 127.1.17.254:39391)
0807 05:09:08.910326 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.910330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.910382 (+ 52us) server_negotiation.cc:407] Connection header received
0807 05:09:08.910503 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.910505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.910546 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.910630 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.911176 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.911822 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.912588 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.912872 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.912967 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.916065 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.916068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.916070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.916090 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.917872 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.918359 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.918362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.918363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.918456 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.918785 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.918790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.918791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.918995 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.919807 (+ 812us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.919876 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.919971 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 05:09:08.920838 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.911525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:43996)
0807 05:09:08.911622 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.911635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.911699 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.911994 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.911997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.912150 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.912226 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.912231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.913139 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.913142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.913654 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.913661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.914132 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.914136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.914217 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.914605 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.914616 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.916454 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.919397 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.919401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.919404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.919644 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.919921 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.919924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.919925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.920016 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.920392 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.920394 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.920479 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.920555 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.920660 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 05:09:08.921408 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.909545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.1.17.253:37471)
0807 05:09:08.909693 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.909699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.909716 (+ 17us) server_negotiation.cc:407] Connection header received
0807 05:09:08.909764 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.909766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.909814 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.909894 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.910383 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.911010 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.912370 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.912730 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.912876 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.916645 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.916649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.916652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.916683 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.918831 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.919423 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.919429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.919433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.919512 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.919839 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.919843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.919844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.920035 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.920100 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.920651 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.920783 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 05:09:08.921468 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.911670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.1.17.252:39043)
0807 05:09:08.911755 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.911770 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.911781 (+ 11us) server_negotiation.cc:407] Connection header received
0807 05:09:08.911829 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.911831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.911873 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.911978 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.912336 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.913033 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.913770 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.914016 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.914130 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.916977 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.916980 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.916983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.917005 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.919250 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.919759 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.919761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.919763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.919834 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.920111 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.920113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.920114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.920297 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.921119 (+ 822us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.921232 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.921333 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 05:09:08.922307 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.909423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.253:37471 (local address 127.0.0.1:48926)
0807 05:09:08.909557 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.909569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.909655 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.909906 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.909908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.910096 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.910206 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.910213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.911122 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.911128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.911911 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.911922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.912876 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.912880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.912993 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.913673 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.913688 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.916443 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.918944 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.918947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.918949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.919308 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.919642 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.919648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.919652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.919735 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.920357 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.920362 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.920453 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.921979 (+ 1526us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.922109 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 05:09:08.923131 24959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.253@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
I20260807 05:09:08.925220 24804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76: Bootstrap starting.
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.17.253@KRBTEST.COM
I20260807 05:09:08.927364 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.917036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:44000)
0807 05:09:08.917164 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.917180 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.917262 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.918388 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.918393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.918645 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.918754 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.918761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.920241 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.920244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.921070 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.921084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.921743 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.921748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.921844 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.922273 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.922292 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.923742 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.925785 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.925788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.925790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.926020 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.926363 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.926366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.926368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.926483 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.926956 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.926959 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.927045 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.927121 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.927233 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 05:09:08.928079 24804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:08.928889 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.917399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.1.17.252:39043)
0807 05:09:08.918127 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.918138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.918154 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:09:08.918190 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.918193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.918242 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.918868 (+ 626us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.918963 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.920131 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.921245 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.921608 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.922051 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.923896 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.923899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.923902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.923925 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.925663 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.926138 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.926141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.926143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.926222 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.926600 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.926604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.926607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.926827 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.928497 (+ 1670us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.928575 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.928689 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":652,"thread_start_us":111,"threads_started":1}
I20260807 05:09:08.929062 24804 log.cc:824] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:08.930631 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.921370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48930 (local address 127.1.17.253:37471)
0807 05:09:08.921477 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.921484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.921751 (+ 267us) server_negotiation.cc:407] Connection header received
0807 05:09:08.921786 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.921788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.921825 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.921891 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.922781 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.923476 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.924516 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.924759 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.924885 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.927355 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.927362 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.927364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.927388 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.928945 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.929421 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.929424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.929426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.929505 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.929808 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.929811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.929813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.930020 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.930200 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.930376 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.930489 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 05:09:08.931664 24880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: Bootstrap starting.
Aug 07 05:09:08 dist-test-slave-1x4g krb5kdc[24719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786079348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.17.252@KRBTEST.COM for kudu/127.1.17.252@KRBTEST.COM
I20260807 05:09:08.934002 24880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:08.934162 1095 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 05:09:08.934604 24804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76: No bootstrap required, opened a new log
I20260807 05:09:08.934904 24880 log.cc:824] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:08.936335 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.923582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.17.252:39043 (local address 127.0.0.1:44012)
0807 05:09:08.924110 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 05:09:08.924123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 05:09:08.924188 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 05:09:08.926738 (+ 2550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 05:09:08.926741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 05:09:08.926896 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 05:09:08.926997 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.927007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.928870 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.928876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.929838 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 05:09:08.929850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.930452 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.930459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 05:09:08.930532 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.931186 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 05:09:08.931199 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 05:09:08.932497 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 05:09:08.934610 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.934614 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.934617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.934916 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.935286 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 05:09:08.935288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 05:09:08.935290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 05:09:08.935384 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 05:09:08.935839 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 05:09:08.935842 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 05:09:08.935926 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 05:09:08.936044 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 05:09:08.936168 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260807 05:09:08.936369 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.923881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.1.17.252:39043)
0807 05:09:08.926441 (+ 2560us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.926448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.926461 (+ 13us) server_negotiation.cc:407] Connection header received
0807 05:09:08.926492 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.926494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.926536 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.926626 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.927636 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.928353 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.929955 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.930268 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.930360 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.932626 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.932628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.932630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.932654 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.934487 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.935095 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.935100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.935102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.935187 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.935487 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.935490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.935491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.935709 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.935807 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.936096 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.936220 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":891,"server-negotiator.queue_time_us":2500,"thread_start_us":74,"threads_started":1}
I20260807 05:09:08.935554 24778 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38132
I20260807 05:09:08.937148 24880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: No bootstrap required, opened a new log
I20260807 05:09:08.938418 24804 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.938627 24804 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2382e3a70fe4014b8ee5ad5adb94f76, State: Initialized, Role: FOLLOWER
I20260807 05:09:08.938921 24804 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.939340 24959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: Bootstrap starting.
I20260807 05:09:08.939396 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.939635 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:08.940153 24804 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:08.940897 24778 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 05:09:08.941064 24778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:08.941752 24880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.941977 24880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e7f3642a277432198fd6d1b8cec7cf2, State: Initialized, Role: FOLLOWER
I20260807 05:09:08.942279 24778 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.942291 24880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.942744 24981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.943029 24981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:08.943512 24880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:08.944223 24854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2e7f3642a277432198fd6d1b8cec7cf2"
I20260807 05:09:08.944480 24854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:08.945830 24854 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2382e3a70fe4014b8ee5ad5adb94f76 in term 1.
I20260807 05:09:08.946244 24752 leader_election.cc:302] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e7f3642a277432198fd6d1b8cec7cf2, c2382e3a70fe4014b8ee5ad5adb94f76; no voters:
W20260807 05:09:08.946375 24982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:08.946467 24982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:08.946543 24970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 05:09:08.946789 24970 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [term 1 LEADER]: Becoming Leader. State: Replica: c2382e3a70fe4014b8ee5ad5adb94f76, State: Running, Role: LEADER
I20260807 05:09:08.947085 24970 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.947681 24959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: Neither blocks nor log segments found. Creating new log.
I20260807 05:09:08.948529 24959 log.cc:824] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: Log is configured to *not* fsync() on all Append() calls
I20260807 05:09:08.948904 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2382e3a70fe4014b8ee5ad5adb94f76. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.948999 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:08.949041 24778 leader_election.cc:288] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [CANDIDATE]: Term 1 election: Requested vote from peers 2e7f3642a277432198fd6d1b8cec7cf2 (127.1.17.253:37471), 02c5e3d9df5a4c12b549ca92fc7589b5 (127.1.17.252:39043)
I20260807 05:09:08.948010 24733 master_runner.cc:186] Error getting master registration for 127.1.17.253:37471: OK, instance_id { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" instance_seqno: 1786079348661961 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:09:08.951570 24733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:09:08.953501 24997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 05:09:08.954411 24997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 05:09:08.955389 24811 master_runner.cc:186] Error getting master registration for 127.1.17.252:39043: OK, instance_id { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" instance_seqno: 1786079348884540 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 05:09:08.958869 24811 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 05:09:08.960184 24854 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [term 1 FOLLOWER]: Refusing update from remote peer c2382e3a70fe4014b8ee5ad5adb94f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:08.960748 24959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: No bootstrap required, opened a new log
I20260807 05:09:08.960968 24970 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:08.965433 24981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2382e3a70fe4014b8ee5ad5adb94f76. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.965629 24981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:08.965916 24959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.966244 24959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5e3d9df5a4c12b549ca92fc7589b5, State: Initialized, Role: FOLLOWER
I20260807 05:09:08.966594 24959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } }
I20260807 05:09:08.968005 24959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 05:09:08.968034 24933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5"
I20260807 05:09:08.968268 24932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 05:09:08.969730 24932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [term 1 FOLLOWER]: Refusing update from remote peer c2382e3a70fe4014b8ee5ad5adb94f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 05:09:08.970469 24994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 05:09:08.971087 24933 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2382e3a70fe4014b8ee5ad5adb94f76 in term 1.
W20260807 05:09:08.971329 24998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:08.971398 24998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 05:09:08.979688 25023 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 05:09:08.980187 25023 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 05:09:08.980047 25012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.980388 25012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:08.980540 25012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2382e3a70fe4014b8ee5ad5adb94f76. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.980702 25012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:08.981912 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.951315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.1.17.253:37471)
0807 05:09:08.951426 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.951431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.965377 (+ 13946us) server_negotiation.cc:407] Connection header received
0807 05:09:08.965969 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.965973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.966028 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.966100 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.967656 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.968387 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.974422 (+ 6035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.974789 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.974864 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.977838 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.977841 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.977844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.977869 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.979845 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.980389 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.980392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.980395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.980485 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.980837 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.980840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.980842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.981087 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.981212 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.981595 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.981731 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 05:09:08.982321 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.951687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.1.17.252:39043)
0807 05:09:08.951784 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.951789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.972415 (+ 20626us) server_negotiation.cc:407] Connection header received
0807 05:09:08.972596 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.972600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.972648 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.972698 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.974643 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.975416 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.976708 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.977088 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.977162 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.978789 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.978793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.978796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.978820 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.980487 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.981017 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.981021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.981023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.981108 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.981392 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.981396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.981397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.981588 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.981731 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.982068 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.982180 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 05:09:08.985836 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:08.951105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.1.17.254:39391)
0807 05:09:08.951239 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:08.951243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:08.965132 (+ 13889us) server_negotiation.cc:407] Connection header received
0807 05:09:08.965598 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:08.965601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:08.965660 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:08.965718 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:08.966888 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.967689 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.973477 (+ 5788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:08.973792 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:08.973874 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:08.981878 (+ 8004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 05:09:08.981881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 05:09:08.981884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 05:09:08.981909 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 05:09:08.983971 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.984464 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.984468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.984469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.984568 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 05:09:08.984917 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 05:09:08.984920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 05:09:08.984921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 05:09:08.985151 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 05:09:08.985254 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:08.985580 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 05:09:08.985692 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 05:09:08.986148 25009 mvcc.cc:205] Tried to move back new op lower bound from 7315781013331935232 to 7315781013290553344. Current Snapshot: MvccSnapshot[applied={T|T < 7315781013331935232}]
I20260807 05:09:08.986647 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.986748 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:08.987437 24981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.987547 24981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:08.993827 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.993942 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76 [sys.catalog]: This master's current role is: LEADER
I20260807 05:09:08.995361 25024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2382e3a70fe4014b8ee5ad5adb94f76" member_type: VOTER last_known_addr { host: "127.1.17.254" port: 39391 } } peers { permanent_uuid: "2e7f3642a277432198fd6d1b8cec7cf2" member_type: VOTER last_known_addr { host: "127.1.17.253" port: 37471 } } peers { permanent_uuid: "02c5e3d9df5a4c12b549ca92fc7589b5" member_type: VOTER last_known_addr { host: "127.1.17.252" port: 39043 } } }
I20260807 05:09:08.995496 25024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 05:09:08.998278 25025 mvcc.cc:205] Tried to move back new op lower bound from 7315781013331935232 to 7315781013290553344. Current Snapshot: MvccSnapshot[applied={T|T < 7315781013331935232}]
I20260807 05:09:08.998833 24997 catalog_manager.cc:1347] Generated new cluster ID: 66e98da3ee5e4e44aaab376a5533583d
I20260807 05:09:08.998894 24997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 05:09:09.016822 24997 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 05:09:09.018066 24997 catalog_manager.cc:1504] Loading token signing keys...
I20260807 05:09:09.027891 24997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2382e3a70fe4014b8ee5ad5adb94f76: Generated new TSK 0
I20260807 05:09:09.028745 24997 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 05:09:09.060848 24807 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.057776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.1.17.254:39391)
0807 05:09:09.057891 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.057895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.059672 (+ 1777us) server_negotiation.cc:407] Connection header received
0807 05:09:09.059905 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.059908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.059954 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.060005 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.060688 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38152: BlockingRecv error: recv got EOF from 127.0.0.1:38152 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 05:09:09.062436 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.058476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.1.17.253:37471)
0807 05:09:09.058603 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.058608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.061424 (+ 2816us) server_negotiation.cc:407] Connection header received
0807 05:09:09.061779 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.061782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.061828 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.061876 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.062324 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48948: BlockingRecv error: recv got EOF from 127.0.0.1:48948 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 05:09:09.063575 24967 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.059072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44034 (local address 127.1.17.252:39043)
0807 05:09:09.059176 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.059180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.061706 (+ 2526us) server_negotiation.cc:407] Connection header received
0807 05:09:09.062870 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.062873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.062916 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.062962 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.063437 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44034: BlockingRecv error: recv got EOF from 127.0.0.1:44034 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 05:09:09.067183 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.065784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.1.17.253:37471)
0807 05:09:09.065869 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.065873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.066294 (+ 421us) server_negotiation.cc:407] Connection header received
0807 05:09:09.066506 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.066509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.066586 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.066638 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.067100 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48962: BlockingRecv error: recv got EOF from 127.0.0.1:48962 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 05:09:09.072046 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.070766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.1.17.253:37471)
0807 05:09:09.070850 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.070854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.071292 (+ 438us) server_negotiation.cc:407] Connection header received
0807 05:09:09.071450 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.071453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.071493 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.071538 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.071968 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48972: BlockingRecv error: recv got EOF from 127.0.0.1:48972 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 05:09:09.077407 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.076010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.1.17.253:37471)
0807 05:09:09.076095 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.076098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.076548 (+ 450us) server_negotiation.cc:407] Connection header received
0807 05:09:09.076764 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.076767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.076841 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.076889 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.077327 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48982: BlockingRecv error: recv got EOF from 127.0.0.1:48982 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 05:09:09.084498 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.083106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.1.17.253:37471)
0807 05:09:09.083228 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.083231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.083736 (+ 505us) server_negotiation.cc:407] Connection header received
0807 05:09:09.083901 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.083904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.083946 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.083991 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.084417 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48998: BlockingRecv error: recv got EOF from 127.0.0.1:48998 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 05:09:09.095712 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.094330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.1.17.253:37471)
0807 05:09:09.094422 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.094426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.094908 (+ 482us) server_negotiation.cc:407] Connection header received
0807 05:09:09.095078 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.095082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.095125 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.095172 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.095630 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49014: BlockingRecv error: recv got EOF from 127.0.0.1:49014 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 05:09:09.115607 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.113929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49022 (local address 127.1.17.253:37471)
0807 05:09:09.114045 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.114050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.114632 (+ 582us) server_negotiation.cc:407] Connection header received
0807 05:09:09.114860 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.114863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.114913 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.114964 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.115492 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49022: BlockingRecv error: recv got EOF from 127.0.0.1:49022 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 05:09:09.151911 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.150270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.1.17.253:37471)
0807 05:09:09.150380 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.150384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.150963 (+ 579us) server_negotiation.cc:407] Connection header received
0807 05:09:09.151159 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.151162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.151213 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.151262 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.151796 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49024: BlockingRecv error: recv got EOF from 127.0.0.1:49024 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 05:09:09.219796 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.218484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.1.17.253:37471)
0807 05:09:09.218646 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.218653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.218875 (+ 222us) server_negotiation.cc:407] Connection header received
0807 05:09:09.219070 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.219073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.219124 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.219215 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.219699 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49034: BlockingRecv error: recv got EOF from 127.0.0.1:49034 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 05:09:09.351418 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.350109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49046 (local address 127.1.17.253:37471)
0807 05:09:09.350271 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.350276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.350455 (+ 179us) server_negotiation.cc:407] Connection header received
0807 05:09:09.350552 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.350554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.350606 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.350677 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.351250 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49046: BlockingRecv error: recv got EOF from 127.0.0.1:49046 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 05:09:09.610638 24885 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 05:09:09.609437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60470 (local address 127.1.17.253:37471)
0807 05:09:09.609598 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:09.609605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:09.609629 (+ 24us) server_negotiation.cc:407] Connection header received
0807 05:09:09.609811 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:09.609815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:09.609877 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:09.609970 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 05:09:09.610512 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60470: BlockingRecv error: recv got EOF from 127.0.0.1:60470 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 05:09:09.972833 24998 catalog_manager.cc:1259] Loaded cluster ID: 66e98da3ee5e4e44aaab376a5533583d
I20260807 05:09:09.972926 24998 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: loading cluster ID for follower catalog manager: success
I20260807 05:09:09.974597 24998 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: acquiring CA information for follower catalog manager: success
I20260807 05:09:09.975854 24998 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e7f3642a277432198fd6d1b8cec7cf2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:09:09.981446 25023 catalog_manager.cc:1259] Loaded cluster ID: 66e98da3ee5e4e44aaab376a5533583d
I20260807 05:09:09.981510 25023 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: loading cluster ID for follower catalog manager: success
I20260807 05:09:09.983167 25023 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: acquiring CA information for follower catalog manager: success
I20260807 05:09:09.984254 25023 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 02c5e3d9df5a4c12b549ca92fc7589b5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 05:09:10.128237 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:10.124735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.1.17.253:37471)
0807 05:09:10.125011 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:10.125015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:10.125031 (+ 16us) server_negotiation.cc:407] Connection header received
0807 05:09:10.125147 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:10.125149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:10.125279 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:10.125384 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:10.125625 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.126128 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.126907 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.127130 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.127204 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:10.127495 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:10.127927 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:10.128016 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:10.128070 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 05:09:10.128115 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":128,"threads_started":1}
I20260807 05:09:10.134793 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:10.131209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.1.17.252:39043)
0807 05:09:10.131450 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:10.131453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:10.131549 (+ 96us) server_negotiation.cc:407] Connection header received
0807 05:09:10.131662 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:10.131665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:10.131772 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:10.131864 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:10.132037 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.132514 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.133347 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.133577 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.133659 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:10.133902 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:10.134393 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:10.134485 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:10.134552 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 05:09:10.134595 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260807 05:09:10.141664 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:10.137679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.1.17.253:37471)
0807 05:09:10.137817 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:10.137822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:10.138060 (+ 238us) server_negotiation.cc:407] Connection header received
0807 05:09:10.138254 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:10.138259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:10.138309 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:10.138374 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:10.138694 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.139442 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.140534 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.140852 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.140959 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:10.141226 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:10.141302 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:10.141365 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:10.141464 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 05:09:10.141525 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 05:09:10.142565 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:10.138096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.1.17.252:39043)
0807 05:09:10.138193 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:10.138198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:10.138553 (+ 355us) server_negotiation.cc:407] Connection header received
0807 05:09:10.138725 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:10.138728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:10.138774 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:10.138824 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:10.139129 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.139693 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.141068 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.141395 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.141499 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:10.141984 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:10.142095 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:10.142142 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:10.142362 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 05:09:10.142436 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 05:09:10.142871 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 05:09:10.137667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.1.17.254:39391)
0807 05:09:10.137927 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 05:09:10.137932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 05:09:10.138098 (+ 166us) server_negotiation.cc:407] Connection header received
0807 05:09:10.138326 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 05:09:10.138332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 05:09:10.138500 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 05:09:10.138661 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 05:09:10.138890 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.139690 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.141057 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 05:09:10.141449 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 05:09:10.141729 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 05:09:10.141868 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 05:09:10.142192 (+ 324us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 05:09:10.142288 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 05:09:10.142644 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 05:09:10.142727 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260807 05:09:10.145954 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24733
I20260807 05:09:10.156083 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24811
I20260807 05:09:10.163743 1095 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXAtKix/build/debug/bin/kudu with pid 24887
2026-08-07T05:09:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3087 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6170 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (282081 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST